Submitted by Zatania t3_xw5hhl in MachineLearning
So I'm doing a thesis paper using BERT and FAISS. Google Colab [haven't tried pro yet] works fine with datasets that are less than 100mb using GPU runtime. But when the dataset is bigger than that, google colab just crashed.
Will colab pro help on this or is there another alternative for this?
Edit: dataset file size that I tried that crashed colab is somewhere around 1gb to 1.5gb.
This… I’m finetuning a large wav2vec2 model on colan with a 1+ GB dataset, no issues here
paperspace is an alternative. they have a free tier too
Their free machines are almost never available, in my experience. Also, all notebooks in their free tier are publicly available, which may be a major downside for some folks.
what is the "sequence length" in BERT ?
The amount of tokens in the input. Sequence length requires quadratic scaling compute.
Pretrained BERT takes in a maximum of 512 tokens.
Yeah! I mean what is the seq_length used by OP :) also the batch size :) I have tried seq_length = 300 but with a small batch size in Colab, specially with AdamW instead of Adam
Random q, why FAISS over ScaNN?
The biggest obstacle to using ScaNN over FAISS is that ScaNN is Linux only.
FAISS can also use the GPU for larger workloads.
For ANN in practice they are close enough.
Are you trying to load the file straight into Colab or are you mounting your Google drive and loading from there?
load straight into colab
like as a test, i downloaded 1 gb dataset from kaggle into colab directly
Try uploading it to your Google Drive first.
Then you can mount your drive in your notebook by using:
from google.colab import drive
drive.mount(“mnt”)
Run the cell and allow access to your Drive when the prompt appears.
In the files tab on the left-hand pane you should now see a folder called mnt listed which will contain the contents of your Google Drive. To get the path to a file you can just right click on the file>copy path.
I'll try this solution if this works, will get back to you.
FYI loading many small files from drive is very slow. If this applies to you, I recommend zipping the files, uploading to drive, copying the zipped file onto your colab machine, and unzipping.
from google.colab import drive
drive.mount('/content/drive')
!cp '/content/drive/My Drive/foo.zip' '/tmp/foo.zip'
os.chdir("/tmp")
!unzip -qq 'foo.zip'
Otherwise, if your dataloader is trying to copy files over from Drive one at a time it's going to be really slow.
Also I'd make sure you're not accidentally loading the entire dataset into RAM (assuming your crash is due to lack of RAM?).
Bad advice
Google Drive access bandwidth is limited and far lower than the Google Colab runtime’s (temporary) HDD storage
Source: worked on training CNN for my bachelor’s thesis
It may actually solve the problem. I’ve run into similar issues before.
Source: CompSci PhD. I use Colab a lot.
[deleted]
Well put files in your google drive and mount your drive
Why don't use Jupyter notebook? If your own machine not good enough you can use AWS or other cloud. It's really cheap like $0.3 per hour for 32gb ram and 8vcpu
what cloud works best that have like a gpu option?
This solution may fit you AWS SageMaker
thanks ill check it out
Even with Colab Pro, you are going to run out of GPU time really quickly. Kaggle's free tier for example gives you more GPU time than Colab Pro does at the moment.
First, I really recommend to go pro+, I am working on colab for two years now and it is usually sufficient for nlp/vision POCs. Later, as projects get more complicated, it is recommended to move to a larger machine (im a master’s student as well and I work on the university cluster, check if there is one at your university, most likely there is one). Second, try maybe to change your training type, and work with generators instead of loading the whole data into memory at once.
Try Intel Devcloud. It is free and you get beefy resources.
Have you tried controlling the batch size (e.g. 32 or 64) and truncate the sequence length to 512?
Kaggle, Paperspace are enough.
[removed]
I regularly use colab for an efficient net and a dataset 30gb big. Though my dataset is properly written in the the tfrecord format, and it is not necessary to load it into memory
Http://Saturncloud.io something like 30 free gpu hours.
Maybe using numpy instead of python list can help
im using numpy
my problem was during the faiss indexing of the embeddings, this is where it crashes
supreethrao t1_ir4ojh3 wrote
You might want to check your data processing pipeline and maybe optimise how you’re allocation GPU RAM / System RAM. Colab pro will help but I’d suggest that try and optimise the way you deal with you data as colab free tier should easily handle datasets in the few GB range