diff options
| author | rowanbeentje <rowan@beent.je> | 2009-09-28 00:43:41 +0000 |
|---|---|---|
| committer | rowanbeentje <rowan@beent.je> | 2009-09-28 00:43:41 +0000 |
| commit | cc0c0a7842e3bff325fa29c71f5115361981797d (patch) | |
| tree | 8c101a961ba86f1f04ddbba00ce76963d9c4f3e3 /UnitTests | |
| parent | 2183eeefefb81846c2cc2c6b4bf68b12167f2b24 (diff) | |
| download | sequelpro-cc0c0a7842e3bff325fa29c71f5115361981797d.tar.gz sequelpro-cc0c0a7842e3bff325fa29c71f5115361981797d.tar.bz2 sequelpro-cc0c0a7842e3bff325fa29c71f5115361981797d.zip | |
Rewrite CSV import:
- Replace the CSV parsing function (arrayForCSV:) with a new SPCSVParser class
- Make speed improvements to SPCSVParser to achieve 1.9x faster parsing than the old arrayForCSV: function
- Rewrite CSV imports to be performed as a streaming import, keeping memory usage much much lower
- CSV field mapping preview is now shown very early on in the import process, as soon as the first hundred rwos are available for a preview
- Progress bars are more consistent and accurate
- CSV rows are grouped into batches of up to 50 (depending on line length) for import, falling back to one-query-per-row if errors occur. The current error reporting level is therefore maintained, but imports of non-erroring data are much much faster.
- Improve processing speed slightly
- Fix some odd edge cases in CSV parsing
This addresses issue #389.
Diffstat (limited to 'UnitTests')
0 files changed, 0 insertions, 0 deletions