nolearn BatchIterator question

Hi all, I just started this competition and already learned a lot the past week. For those using nolearn in this competition, did you run into out of memory issues when trying to load the CSV file produced by the mxnet Python script? I ended up running out of memory on my AWS spot instance so I thought to try out subclassing the BatchIterator and load the single CSV in batches. This seemed to work as far as the iterator implementation is concerned but doesn’t seem to fit well with how nolearn is generating the Train/Validation split. Mxnet uses two CSV iterators but I’m not sure if nolearn supports this type of input. I’m sure it’s possible to change the internals of nolearn a bit, but I don’t think I…


Link to Full Article: nolearn BatchIterator question