A set of scikit-learn-style transformers for encoding categorical variables into numeric by means of different techniques.
The package requires:
To install the package, execute:
$ python setup.py install
pip install category_encoders
conda install -c conda-forge category_encoders
To install the development version, you may use:
pip install --upgrade git+https://github.com/scikit-learn-contrib/category_encoders
All of the encoders are fully compatible sklearn transformers, so they can be used in pipelines or in your existing scripts. Supported input formats include numpy arrays and pandas dataframes. If the cols parameter isn't passed, all columns with object or pandas categorical data type will be encoded. Please see the docs for transformer-specific configuration options.
There are two types of encoders: unsupervised and supervised. An unsupervised example:
from category_encoders import * import pandas as pd from sklearn.datasets import load_boston # prepare some data bunch = load_boston() y = bunch.target X = pd.DataFrame(bunch.data, columns=bunch.feature_names) # use binary encoding to encode two categorical features enc = BinaryEncoder(cols=['CHAS', 'RAD']).fit(X) # transform the dataset numeric_dataset = enc.transform(X)
And a supervised example:
from category_encoders import * import pandas as pd from sklearn.datasets import load_boston # prepare some data bunch = load_boston() y_train = bunch.target[0:250] y_test = bunch.target[250:506] X_train = pd.DataFrame(bunch.data[0:250], columns=bunch.feature_names) X_test = pd.DataFrame(bunch.data[250:506], columns=bunch.feature_names) # use target encoding to encode two categorical features enc = TargetEncoder(cols=['CHAS', 'RAD']) # transform the datasets training_numeric_dataset = enc.fit_transform(X_train, y_train) testing_numeric_dataset = enc.transform(X_test)
For the transformation of the training data with the supervised methods, you should use
fit_transform() method instead of
fit().transform(), because these two methods do not have to generate the same result. The difference can be observed with LeaveOneOut encoder, which performs a nested cross-validation for the training data in
fit_transform() method (to decrease over-fitting of the downstream model) but uses all the training data for scoring with
transform() method (to get as accurate estimates as possible).
Furthermore, you may benefit from following wrappers:
Additional examples and benchmarks can be found in the
Category encoders is under active development, if you'd like to be involved, we'd love to have you. Check out the CONTRIBUTING.md file or open an issue on the github project to get started.