streamlit_rag

A streamlit app for interfacing with a local LLM.

• Install required libraries. The LLM and RAG system relies on two key libraries you should set up and make sure are working independently:
  • nlp_pipeline for the processing of documents
  • local_rag_llm for the LLM itself
• Download the LLM (in .gguf format) you would like to use and put it in the models/ directory.
• Update the metadata/llm_list.csv file with the URL and local path of the model (mistral-docsgpt is recommended for RAG)
• If you would like to prepopulate a corpus outside of the app, add its name to the metadata/corpora_list.csv file in the name the file, the directory to the .txt files in the text_path column, and the path to the metadata file in the metadata_path column. The metadata file can contain anything, but must at least include a text_id column (unique identifier starting from 1) and a file_path column, containing the absolute path of all the text files in the corpus.
• In the metadata/user_list.csv file, put user names and emails. If the app is in use, it will tell you by who.
• run the app from the command line with streamlit run app.py --server.port 8*** at whatever port you wish.
• To get the app online quickly, you can use ngrok to expose this local port to be able to access via the internet.

• The app has a password, which you can set by creating a .streamlit/ directory in the base directory of the app, with a secrets.toml file inside containing password = "desired_password"
• You can change various theme options by creating a .streamlit/config.toml file, containin e.g.:

[theme]
primaryColor="#5b92E5"
backgroundColor="#FFFFFF"
secondaryBackgroundColor="#F0F2F6"

• The app will dynamically calculate the maximum number of users, which is automatically determined by the RAM of your GPU or Apple silicon Mac and the models you are using. On CPU, it will be limited to 1 concurrent user, though you can change this by editing the calc_max_users() function in the helper/user_management.py file.
• Users' sessions will be reserved for 3 minutes by default after each interaction they have. If it is left idle longer than that, they become boot eligible if another user tries to log on. The user can change the inactive reserve period by editing the Lockout duration dropdown selection.
• A user can guarantee access by setting an override password in .streamlit/secrets.toml, which if entered instead of the normal password will boot the user who last used the application longest ago regardless of their reservation period.
• Add additional users by editing the metadata/user_list.csv file.

• Parameters are explained in the sidebar in their respective tooltips.
• Change their values then hit the Reinitialize model button to reinitialize the model with those parameters.
• The two parameters under Vector DB parameters require the recreation of the vector database, which may take longer if you have a very large corpus. If you change either of these, click the Reinitialize model and remake DB button instead.
• Hit the Reset model's memory button to clear the model's short-term memory/context.

• The system intializes with no corpus, so you are chatting with the vanilla LLM

• To query over your own documents, you have 6 options:

  1. Preprocess your files into .txts and place in the appropriate places according to the instructions in the "Set up" section. The corpus will then appear as an option under the Which corpus selector.
  2. Paste a comma-separated list of URLs into the URLs box. Make sure these URLs aren't behind a log in/paywall. If that is the case, copy the content to a Word or .txt file and upload directly.
  3. Upload/drag a single .csv, .doc, .docx, .pdf, or .txt file into the Upload your own documents box
  4. Upload a single metadata.csv file into the Upload your own documents box. The CSV can include any metadata you want, but must at least include a web_filepath column pointing to the website or PDF file online.
  5. Upload a .zip file containing many documents. Put all your documents into a directory called corpus/, then zip it. Upload that file into the Upload your own documents box.
  6. Upload a .zip file containing many documents as well as a metadata file. Put all your documents into a directory called corpus/, then put a file called metadata.csv at the same level as the corpus/ directory (not in directory), then zip the directory and CSV together. The CSV needs to have at least a column named filename with the filename of the documents. Upload that file into the Upload your own documents box.

• You can persist your corpus if it is large by typing a name other than temporary to the Uploaded corpus name box. This name will then appear as an option under the Which corpus dropdown. It should be lower case with no spaces or special characters, use underscores for spaces.

• Then hit the Process corpus button. This will both process the corpus and then reinitialize the model on this corpus, wait for both to finish.

• You can clear out old corpora from local files and the database by using helper/clear_corpus.py. E.g., run the command line in the helper/ directory, then enter:

  • python clear_corpus.py --keep corpus1,corpus1 to delete everything except corpus1 and corpus2
  • python clear_corpus.py --remove corpus1,corpus2 to remove only corpus1 and corpus2