The PG-Adaptor package is a collection of tools designed to manage and automate the data handling process for the Press Information Bureau (PIB) feedback system. It is structured into modules corresponding to specific roles including data harvesting, data processing, and alert dispatching.
Follow these instructions to get a copy of the project up and running on your local machine for development and testing purposes.
Before you begin, ensure you have the following installed:
- Python 3.6 or higher
- PostgreSQL
- Psycopg2
- Python-dotenv
- Clone the repository
git clone https://github.com/your-username/PG-Adaptor.git
cd PG-Adaptor-
Environment Configuration
Create or use a .env file in the root of your project directory and add your database credentials and other configuration settings:
HOSTNAME1=your_database_host
PORT=your_database_port
USERNAME1=your_database_username
PASSWORD=your_database_password
DATABASE=your_database_nameThe package contains several modules, each with functions related to specific roles:
data_harvesters.py: Functions to insert raw data into the source schema.data_alchemists.py: Functions to insert processed data into the processed_and_analyzed schema.alert_dispatchers.py: Functions to manage notifications in the Notification schema.
Examples
To use a function from the data_harvesters module:
from pg_adaptor.data_harvesters import insert_website_data
# Example data to insert
data = [
('http://example.com', '<html>...</html>', 'English'),
# ... more data
]
insert_website_data(data)Contributions are welcome! To contribute to this project, please follow these steps:
-
Fork the repository.
-
Create a new branch for your feature or bug fix.
-
Make your changes and commit them.
-
Push your changes to your fork.
-
Create a pull request to the main repository.
Please make sure to follow the code of conduct and contribution guidelines.
If you have any questions, suggestions, or feedback, please don't hesitate to contact us at aashishnkumar@gmail.com.
This project is licensed under the MIT License. See the LICENSE file for details.