aboutsummaryrefslogtreecommitdiffstats
path: root/Source/TableDump.m
Commit message (Collapse)AuthorAgeFilesLines
* Rewrite CSV import:rowanbeentje2009-09-281-571/+435
| | | | | | | | | | | | | - Replace the CSV parsing function (arrayForCSV:) with a new SPCSVParser class - Make speed improvements to SPCSVParser to achieve 1.9x faster parsing than the old arrayForCSV: function - Rewrite CSV imports to be performed as a streaming import, keeping memory usage much much lower - CSV field mapping preview is now shown very early on in the import process, as soon as the first hundred rwos are available for a preview - Progress bars are more consistent and accurate - CSV rows are grouped into batches of up to 50 (depending on line length) for import, falling back to one-query-per-row if errors occur. The current error reporting level is therefore maintained, but imports of non-erroring data are much much faster. - Improve processing speed slightly - Fix some odd edge cases in CSV parsing This addresses issue #389.
* • if the user chose to open a SQL file which is larger than 1MB SP asks ↵Bibiko2009-09-261-0/+5
| | | | for confirmation, furthermore if a connection is available the user can choose 'Import' instead of loading it into the Query Editor for cases that an user invoked 'Open…' accidentally instead of 'Import…'
* - Fix an SQL view export error - some MySQL server versions allow NOT NULL ↵rowanbeentje2009-09-141-1/+5
| | | | DEFAULT NULL, but don't allow definition of tables as that - fix the export of temporary placeholder tables for those views. Thanks to Andreas Falk for report and details.
* Significantly improve export:rowanbeentje2009-09-141-182/+314
| | | | | | | | - Rework CSV export to stream data, significantly reducing memory consumption and so increasing speed and stability when exporting large tables. By default safe/fast streaming is used, but a checkbox is available to select "low memory mode" full streaming, allowing export of any size table in theory. This addresses Issue #224. - Rework XML export to stream data in the same way, also significantly reducing memory usage and providing the option of using low memory mode. - Make SQL, CSV and XML export progress bars update more smoothly - When exporting the current browse view or custom query result, show an indeterminate progress bar when copying large resultsets to avoid the app appearing to hang
* • fixed bug for exporting tables as XML: now the very last row will be ↵Bibiko2009-09-111-1/+3
| | | | | | written to file [ this fixes issue 404 ] • FieldEditorSheet: if string cell content is "NULL" select the entire string for convenience
* - Re-enable the fine-grained query logging preferencesrowanbeentje2009-08-311-1/+22
| | | | | | | - Add defaults for fine-grained logging preferences - Add a method to TableDocument to allow setting the query mode, and use the query mode to control logging - Set import/export and custom query to set the appropriate query modes
* Refactor CSV/SQL import structure slightly, and rewrite SQL import:rowanbeentje2009-08-311-298/+413
| | | | | | | - SQL import now reads and processes files in full streaming mode, running queries as they are encountered - Memory usage during import is significantly reduced, and should stay within a few megabytes; the significant memory use remaining is for query logging - The progress bar more accurately represents progress and is shown at once (this addresses Issue #320)
* Update Localizable.strings by running genstrings and remove use of multiple ↵stuconnolly2009-08-281-3/+3
| | | | comments for a single string.
* - Change MCPStreamingResult to use a safer streaming mode by default - ↵rowanbeentje2009-08-201-1/+1
| | | | | | | download all results as fast as possible from the server, to avoid blocking, but do so in a background thread to allow results processing to start as soon as data is available. Many thanks to Hans-Jörg Bibiko for assistance with this. - Add an option to the SQL export dialog to allow selection of the full-streaming method, with a warning that it may block table UPDATES/INSERTS.
* Issue 351: Export CSV pipebamse162009-08-181-16/+8
| | | | | | I had to cheat and name the file %@.csv, so the suggested filetype is csv, but you can change it to something else.
* - Fix very end of exported SQL for tabel content with new export coderowanbeentje2009-08-171-1/+1
|
* - Fix field names in new import following classic last-minute change...rowanbeentje2009-08-171-1/+1
|
* Rework SQL export:rowanbeentje2009-08-171-12/+30
| | | | | | | - Added an MCPStreamingResult class to MCPKit, to allow streaming results from the server including fast array access of each row - Tweak SQL export to use the streaming result class and to keep memory usage lower End result is generally faster exports, more accurate progress bars, and much much lower (and consistent) memory usage.
* Rework data loading in TableContent:rowanbeentje2009-08-111-1/+1
| | | | | | | | | - Data loading now only occurs in one place in the code. This improves consistency and fixes a number of actions which used to trigger a full table reload followed instantly by a filter when the action was performed - If "Reload data after..." prefs are unticked, no longer load the data (ie the preference now works) - Make table count text more consistent and useful - Fix a number of small position-saving type problems with filters and limits active. This fixes Issue #200. - Clean up and standardise the code dealing with data storage - only one data storage array is now used.
* Source tidy up and missing SVN properties.stuconnolly2009-08-071-15/+16
|
* Fix sortorder of tables in export multiple tables as CSV/XML accesory viewbamse162009-08-051-3/+15
|
* Overhaul CSV import functionality:rowanbeentje2009-08-021-116/+244
| | | | | | | - When selecting CSVs SP will attempt to auto-detect the line endings to use - Throw an error if the CSV to be imported appears to have more than 512 columns, usually due to wrong line ending/quote/etc selection - Rewrite the CSV parser completely. New version correctly deals with CSV line terminators which are escaped or in enclosed content, correctly deals with quote strings which are the same as escape strings, and fixes a number of small edge cases. Performance on very long complex strings is slightly slower (~1.5 slower on long strings) but on large but simple tables is faster (~2.2x faster); memory usage is ~1.3x as high but all autoreleased. This addresses Issue #252.
* The size of the import array only needs to be counted once.stuconnolly2009-07-311-5/+6
|
* - Fix "endSheet:returnCode: requires a non-nil sheet" log lines during importsrowanbeentje2009-07-281-12/+6
| | | | | | - Enable threaded animation for import/export progress bars as it has minimal overhead and looks nicer - Fix exceptions caused by tablesList drawing during imports
* Improve TablesList significantly:rowanbeentje2009-07-281-6/+3
| | | | | | | | - If there are twenty or more tables, show a table quicksearch/filter at the top of the list, and update the rest of the code to match. This addresses issue #178. - Select tables and views alphabetically by user's current locale (instead of default MySQL "A B C a b c") - When adding or duplicating tables, insert them at the correct point - Fix a number of minor display bugs caused by incorrect interaction with the tables list caches
* - Fix a reproducible crash when exporting tables caused by a string being ↵rowanbeentje2009-07-211-3/+5
| | | | autoreleased; this addresses Issue #341
* Merge framework integration branch back to trunk. Summary of changes:stuconnolly2009-07-211-14/+14
| | | | | | | | | | | | | | | - Includes all custom code from subclasses CMMCPConnection and CMMCPResult, meaning they have subsequently been removed from the project. - All previous Sequel Pro specific code in the above subclasses has been removed in favour of the delegate (currently set to TableDocumet) informing the framework of such information. - All references to CMMCPConnection and CMMCPResult have subsequently been changed to MCPConnection and MCPResult. - Framework includes MySQL 5.1.36 client libraries and source headers. - Framework is now built as a 4-way (32/64 bit, i386/PPC arch) binary. - All import references to <MCPKit_bundled/MCPKit_bundled.h> have been changed to <MCPKit/MCPKit.h>. - New script 'build-mysql-client.sh' can be used to build the MySQL client libraries from the MySQL source. See the script's header for a list of available options or run it with no arguments to display it's usage. Note that there are still a few changes to be made to the framework with regard to removing Sequel Pro specific calls to the delegate. These however can be made later on as they have no effect on functionality and are merely design changes. Also, note that any future development done on the framework should be made to be as 'generic' as possible, with no Sequel Pro specific references. This should allow the framework to be integrated into another project without the need for SP specific code.
* - Make the DBView window the document window. This allows the document to ↵rowanbeentje2009-07-151-5/+5
| | | | | | | | | | be closed when the window is closed, freeing the document's memory - Update a number of dealloc methods to include more retained memory, and to avoid releasing non-retained memory - Remove notification observers and delegates where appropriate to avoid issues after document closing - Fix a couple of memory leaks - Support window cascading for all windows past the first, using the first window as the autosave window
* - Fix placeholder exports of views containing enums, improving on r959rowanbeentje2009-07-071-1/+7
|
* - Update the import/export progress sheet title to reflect the current activityrowanbeentje2009-07-061-2/+25
| | | | | - Fix mutliple-table CSV and XML export when a view is selected - data for the view is now correctly exported
* - Fixes crashes in the CSV Field Mapping sheet caused by multiple threads ↵rowanbeentje2009-07-061-5/+7
| | | | | | | and autorelease pools (fixes Issue #326) - Reset the field mapping table list whenever it's about to be populated, fixing errors when performing imports into multi databases in the same session
* - Correctly SQL export views with interdependencies on other views or ↵rowanbeentje2009-06-271-1/+78
| | | | tables, resolving Issue #313
* • queryString: code cleaning and processing each encoding equallyBibiko2009-06-191-20/+25
| | | | | | | | - Since we are using mysql_real_query() there is no need to generate a \0 terminated cString. - The disadvantage of using UTF8String is that it returns NULL if the encoding fails. - To process each encoding equally we can now use [NSString dataUsingEncoding:enc allowLossyConversion:lossy]. In oder to avoid the overhead while calling it the following inline function is introduced: NSStringDataUsingLossyEncoding(aStr, enc, lossy) := [aStr dataUsingEncoding:enc allowLossyConversion:lossy] • import of CSV: code cleaned and optimized for speed a little
* • added to SPArrayAdditions.h: NSArrayObjectAtIndex() inline functionBibiko2009-06-181-35/+34
| | | | | | | | | - id o = NSArrayObjectAtIndex(anArray, index) :== id o = [anArray objectAtIndex:index] - this speed up it ~3µs per call - replaced that inline function for such calls within loops to speed up them • used IMP function pointers for keepAlive calls within queryString: • set -O3 (Fastest) compiler option • allow in preference pane "Tables" to set the Limit up to 50000
* updated dot generator to handle multi-col fk's with a different arrow type ↵mtvee2009-06-111-4/+19
| | | | and views as a different header color
* update dot file genertor to use SPTableData instead of a v5 style query for ↵mtvee2009-06-101-19/+21
| | | | fk relations
* • reload table list, database pull-down menu according to user's ↵Bibiko2009-06-051-30/+48
| | | | | | | | | statements in the Custom Query editor if necessary - if statement begins with: use, create, alter, rename, drop • reload table list, database pull-down menu according to imported statements • sped up "Import MySQL Dump" • fixed some tiny issues of the last commit
* Fixed some memory leaks found using llvm/clang. There are still some to fixbamse162009-06-051-7/+10
|
* - visual improvements to the graphviz dot file outputmtvee2009-06-051-7/+6
|
* - added schema export to basic graphviz dot filemtvee2009-06-051-0/+141
|
* • added support for the “delimiter” commandBibiko2009-06-041-2/+2
| | | | | | | | | | | | | | - added (NSArray *) splitSqlStringByCharacter:(unichar)character; (NSArray *) splitSqlStringIntoRangesByCharacter:(unichar)character; (long) firstOccurrenceInSqlOfCharacter: to the SQLParser which recognize a “delimiter” command • queryAtPosition now works with ranges to speed it up - the current query ranges resp. the just activated query range are cached in order to avoid parsing if the user only navigates through the textView buffer, or if the user calls Run Prev/Current Query only • the "import dump" function makes usage of that new “delimiter” support - i.e. dumps with procs/funcs declaration could be imported IMPORTANT: Please check the new SQLParser exhaustively in order to prove that new approach
* • some minor code cosmeticsBibiko2009-05-191-5/+8
|
* More header updates for source files, including Subversion Id property.stuconnolly2009-05-191-1/+2
|
* Issue 233: Option to copy field headings from result panelsbamse162009-05-051-15/+6
| | | | | | Also replaced some NSLog with DLog/ALog
* • ADDED createViewSyntaxPrettifier method to a NSString to make the syntax ↵Bibiko2009-04-231-1/+1
| | | | | | a bit more readable - used for show/copy create view syntax as well as for a MySQL dump
* - Fixed an issue causing a crash with large SQL imports. NSLog was dumping ↵avenjamin2009-04-221-2/+6
| | | | the sql file and was crashing due to the size of the file. Line is commented out in non-Debug builds.
* - Update delete table and database warning messages to be more user friendly ↵stuconnolly2009-04-171-2/+2
| | | | | | | | (issue #192). - Also make these messages have a style of critical to indicate the potential loss of data as a result of performing the operation. - Updated Localizable.strings to accommodate new dialog messages.
* Fix for issue #173. Change export filename date format to YYYY-MM-DD and ↵stuconnolly2009-04-151-2/+2
| | | | remove '_dump' and spaces from filename.
* - part 4 of merge from 'avenjamin' branch into trunk.avenjamin2009-04-101-7/+61
| | | | - committing Source
* - Second part of r498: ensure that SQL imports are read in as UTF8, and ↵rowanbeentje2009-04-081-5/+25
| | | | queries are sent as UTF8, if possible; this correctly preserves encoding data when working with files export from Sequel Pro or via mysqldump, and attempts to fall back to the current connection encodings for other files. This should resolve Issue #116.
* - Select 'drop tables' option for MySQL export by default, to improve ↵rowanbeentje2009-04-061-19/+46
| | | | | | | backup-type process and more closely match mysqldump, and correctly drop views - Consistently set and restore the encoding appropriately for SQL import/export, and export to UTF8 files to correctly store all characters. Goes a good way towards addressing Issue #116.
* - When exporting to an SQL file, disable foreign key checks, change SQL ↵rowanbeentje2009-03-281-11/+41
| | | | | | | mode, and alter notes setting, and disable unique checks if appropriate. Dumps generated by Sequel Pro will now be able to be imported even if foreign keys etc are present, without throwing errors and aborting import. - When exporting to an SQL file, wrap the data import commands with table locking and key disabling commands - improves import speeds.
* - The status tab now no longer crashes when viewing Views. (Resolves Issue ↵rowanbeentje2009-03-281-18/+29
| | | | | | | #212) - SQL dump now correctly exports the CREATE ALGORITHM statement for views, and no longer attempts to output data for views. Databases with views can now be exported/imported without errors.
* Fixed bug where exporting current table would use the field terminator, ↵avenjamin2009-03-271-13/+25
| | | | enclosure, escape and line ending characters from the "export multiple tables" dialog instead.
* - fixed issue #203 (backticks in identifiers not supported)jakob2009-03-241-11/+13
| | | | | | | | - added a backtickQuotedString: method to SPStringAdditions - created the file SPArrayAdditions for a componentsJoinedAndBacktickQuoted: method In the future, we should use backtickQuotedString: to quote identifiers like this: [NSString stringWithFormat:@"SELECT * FROM %@", [tableName backtickQuotedString]]
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-02 18:26:08 +0000