ASKGPT - Chatbot Application

ASKGPT is a chatbot application similiar to ChatGPT where, I have used Falcon-7b pretrainned model for question answering tasks.

System Architecture

Component Defination

CLIENT

Client Service is simply a server-side rendering react application to serve the purpose of user interface.

It is a NextJS Application.

AUTH Service

This service take care of user information and handles sessions for user interaction.

It is a NodeJS Application.

COMMUNICATION Service

This service is an important service which stores chats and communicate with CHATBOT Service for answer generation.

It is a NodeJS Application. Here, for communication between different service NATS service is used.

REPORT Service

This service is used to report the response which is generated by CHATBOT Service. This service takes correct response from user corresponding to incorrect response of CHATBOT Service.

The Chatbot model is further trained using these data afterwards.

It is a NodeJS Application. Here, for communication between different service NATS service is used.

CHATBOT Service

This service handles answer generation corresponding to user input. Here, Falcon-7B pretrainned model is using which is fine tunned using LoRA on "OpenAssistant/oasst1" chatbot dataset.

This service communicate ith other service though NATS Streaming Server.

TODO Tasks

• Creating service for schedule model training and model validation ith auto deployment.
• Completing Client Service for better UIUX experience with server-side rendering.

Installation Info (locally):

Requirement:

Docker and Kubernetes should be installed and configured. (Also, required ingress-nginx installed for Windows and MacOS users and enabled in Ubuntu)

• Clone the project and install it locally by using

  git clone https://github.com/Ishan25j/ASKGPT-App.git

• Go to every node services folder (i.e auth, client, communication-srv, and report) and install required package using

  npm install

Note: Common folder is the submodule which orginally shared the common source code between each services which has been published on npm websites. Nothing that much important if you delete that submodule.

• After testing and installing package make you you have ingress-nginx enable on your system. Now go to you host file

In Windows: C:\Windows\System32\Drivers\etc\hosts In Linux\MacOS: \etc\hosts Edit it at the bottom add 127.0.0.1 askgpt.dev and save it

• If you are using minikube then run minikube ip And use that ip instead of 127.0.0.1 (i.e. minikube_ip eventure.dev)

So, this edit tries to run eventure.dev locally instead of finding it online via browser

Now after completing all the above steps and configuration, let's run the application.

Running it locally:

• Now, you need to create three kubernetes secrets in order to run this. So just edit the three given commands values as per the requirement

• For JWT:

  kubectl create secret generic jwt-secret --from-literal=JWT_KEY=<'JWT key you want to keep for auth'>

• For PERF Model:

  kubectl create secret generic perf-model-secret --from-literal=PERF_MODEL_KEY=<'enter perf model id'>

• For AutoTokenizer:

  kubectl create secret generic autotoken-model-secret --from-literal=AUTOTOKEN_MODEL_KEY=<'enter autotokenizer used in model'>