supreethrao t1_jdy0xtd wrote on March 28, 2023 at 12:43 AM

Reply to [D]Suggestions on keeping Llama index cost down by darkbluetwilight

Hi , to address Update2 , I think you’ll have to change your prompt to GPT3.5-turbo significantly. LLama index also has a cost estimator function that assumes a dummy LLM backend and calculates the expected cost , you can also use OpenAI’s tokenizer called “tiktoken” which is available on GitHub to calculate the exact number of tokens your text produces

supreethrao t1_jdv1whe wrote on March 27, 2023 at 12:30 PM

Reply to [D]Suggestions on keeping Llama index cost down by darkbluetwilight

Hi, there’s already support for ‘gpt-3.5-turbo’ in llama index , the examples can found in the git repo . You can also switch for SimpleVectorIndex to a TreeIndex , this could lower your cost

supreethrao t1_ir4ojh3 wrote on October 5, 2022 at 9:29 AM

Reply to [R] Google Colab alternative by Zatania

You might want to check your data processing pipeline and maybe optimise how you’re allocation GPU RAM / System RAM. Colab pro will help but I’d suggest that try and optimise the way you deal with you data as colab free tier should easily handle datasets in the few GB range