Wednesday, 14 August 2013

Huge text file to small excel files

Huge text file to small excel files

I have a huge text file (4 GB), where each "line" is of the syntax:
[number] [number]_[number].
For example
123 12_14
1234 13_456
33 12_12
24 678_10
My purpose is to have this data saved as Excel file, where each "line" in
the text file,
is a row in the excel file. According to the past example:
[A1] 123
[B1] 12_14
[A2] 1234
[B2] 13_456
[A3] 33
[B3] 12_12
[A4] 24
[B4] 678_10
My plan is to iterate the text "lines", as advised here, separate the
"lines",
and save to the cells in an excel file.
Because of the text size issue, I thought to create many small excel
files, which all together will be equal to the text file. Finally, I will
analyze the small excel files (things like counting stuff...), using
python again to do so.
I'm considering the best way to create and analyze the excel files.
As mentioned here the main libraries are xlrd and csv.

No comments:

Post a Comment