Merge branch 'master' of github.com:minerva-ml/open-solution-home-credit

Update README.md

Dev (minerva-ml#96)

* Smart features (minerva-ml#61)

* Update README.md

* Update README.md

* Update

* Smart features update

* More descriptive transformer name

* Reading all data in main

* More application features

* Transformer for cleaning

* Multiinput data dictionary

* Fix (minerva-ml#63)

* fixed configs

* dropped redundand steps, moved stuff to cleaning, refactored groupby (minerva-ml#64)

* dropped redundand steps, moved stuff to cleanining, refactored groupby

* restructured, added stacking + CV

* Fix format string

* Update pipeline_manager.py

clipped prediction -> prediction

* added stratified kfold option (minerva-ml#77)

* Update config (minerva-ml#79)

* dropped redundand steps, moved stuff to cleanining, refactored groupby

* restructured, added stacking + CV

* Update pipeline_config.py

* Dev review (minerva-ml#81)

* dropped feature by type split, refactored pipleine_config

* dropped feature by type split method

* explored application features

* trash

* reverted refactor of aggs

* fixed/updated bureau features

* cleared notebooks

* agg features added to notebook bureau

* credit card cleaned

* added other feature notebooks

* added rank mean

* updated model arch

* reverted to old params

* fixed rank mean calculations

* ApplicationCleaning update (minerva-ml#84)

* Cleaning - application

* Clear output in notebook

* clenaed names in steps, refactored mergeaggregate transformer, changed caching/saving specs (minerva-ml#85)

* local trash

* External sources notebook (minerva-ml#86)

* Update

* External sources notebook

* Dev lgbm params (minerva-ml#88)

* local trash

* updated configs

* dropped comment

* updated lgb params

* Dev app agg fix (minerva-ml#90)

* dropped app_aggs

* app agg features fixed

* cleaned leftovers

* dropped fast read-in for debug

* External_sources statistics (minerva-ml#89)

* Speed-up ext_src notebook

* exernal_sources statistics

* Weighted mean and notebook fix

* application notebook update

* clear notebook output

* Fix auto submission (minerva-ml#95)

* updated best model name

* changed best model path

* corrections

Dev solution 2 (minerva-ml#60)

* dev-Solution1 (minerva-ml#16)

* solution 1

* Add features

* random-search (minerva-ml#17)

* random-search

* Update neptune.yaml

* Solution 1 - upgrade (minerva-ml#18)

* Remove save_eval function

* Add more features

* Refactor code

* Add groupby aggregation

* Add numerical_features to groupby_aggregation

* Add AGGREGATION_RECIPIES

* Refactor code - steppy 0.1.1 (minerva-ml#22)

* Refactor code

* Include suggested changes

* Update models.py

* Verifying submittion (minerva-ml#26)

* Refactor code

* Do not save submission if dev_mode

* Verifying submittion

* Add USELESS_COLUMNS

* Include remaining features

* Fix random search

* added requirements

* optimized imports

* Fix solution-1 (minerva-ml#38)

* Refactor code

* Fix

* small refactor

* added neptune-cli to requirements

* added steppy-toolkit to requirements

* one line clip in postprocessing (minerva-ml#39)

* Kaggle api (minerva-ml#42)

* dev-Solution1 (minerva-ml#16)

* solution 1

* Add features

* random-search (minerva-ml#17)

* random-search

* Update neptune.yaml

* Solution 1 - upgrade (minerva-ml#18)

* Remove save_eval function

* Add more features

* Refactor code

* Do not save submission if dev_mode

* Verifying submittion

* Add USELESS_COLUMNS

* Include remaining features

* Fix random search

* kaggle-api

* Refactor code - steppy 0.1.1 (minerva-ml#22)

* Refactor code

* Include suggested changes

* Update models.py

* Verifying submittion (minerva-ml#26)

* Refactor code

* Do not save submission if dev_mode

* Verifying submittion

* Add USELESS_COLUMNS

* Include remaining features

* Fix random search

* added requirements

* optimized imports

* Fix solution-1 (minerva-ml#38)

* Refactor code

* Fix

* small refactor

* added neptune-cli to requirements

* Fix

* Fix 2

* Update kaggle-message

* Update neptune.yaml

* Fix neptune_random_search.yaml

* Update neptune.yaml

* Update neptune_random_search.yaml

* Update main.py

* Update main.py

* modify neptune.yaml and pipeline_config.py (minerva-ml#45)

* dev-Solution1 (minerva-ml#16)

* solution 1

* Add features

* random-search (minerva-ml#17)

* random-search

* Update neptune.yaml

* Solution 1 - upgrade (minerva-ml#18)

* Remove save_eval function

* Add more features

* Refactor code

* Modify neptune.yaml

* Modify pipeline_config.py

* Modify

* Update feature_extraction.py

* small corrections

* Add features from bureau.csv

* Extract filename from path

* Random forest implementation

* Yaml files update

* Log reg implementation

* small corrections, ToNumpyLabel moved to utils

* Some code cleaning

* Normalizer implementation

* TargetEncoding replaced by OrdinalEncoding

* SVC added, yaml files update

* Category encoding (minerva-ml#48)

* TargetEncoding replaced by OrdinalEncoding

* Typo fix

* corrections

* small changes in RandomSearchOptimizer

* refactored pipeline_config, added RANDOM_SEED const, added data shuffle param in both yamls

* removed stratified split, log loss row, save_eval is parameter_eval, updated reqs, optimized imports, refactored to work with steppy >= 0.1.2,

* refactored pipeline_config, moved LightGBM to steppy-toolkit, added logging to main.py and changed its structure

* prepared yamls

* small refactors

* bug fix in hyperparam tuning, changed structure in pipelines.py

* correction

* moved pipeline blocks to separate file

* added XGBoost to the pipelines, small refactors

Dev solution 1 (minerva-ml#40)

* dev-Solution1 (minerva-ml#16)

* solution 1

* Add features

* random-search (minerva-ml#17)

* random-search

* Update neptune.yaml

* Solution 1 - upgrade (minerva-ml#18)

* Remove save_eval function

* Add more features

* Refactor code - steppy 0.1.1 (minerva-ml#22)

* Refactor code

* Include suggested changes

* Update models.py

* Verifying submittion (minerva-ml#26)

* Refactor code

* Do not save submission if dev_mode

* Verifying submittion

* Add USELESS_COLUMNS

* Include remaining features

* Fix random search

* added requirements

* optimized imports

* Fix solution-1 (minerva-ml#38)

* Refactor code

* Fix

* small refactor

* added neptune-cli to requirements

* added steppy-toolkit to requirements

* one line clip in postprocessing (minerva-ml#39)

* cleaning code for realease of the solution-1

* cleaning code for the solution-1

* cleaning code for the solution-1 (again)