| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
| |
background thread, and integrate for SQL import:
- Implement streaming reading of gzip-compressed files for SQL import
- Support exporting SQL dumps into a gzip-compressed file
- SPFileHandle supports the most-used subset of NSFileHandle commands for easy integration
- Integrate zlib 1.2.4 for improved gzip streaming performance (and support for custom buffer sizes and file offset positions)
This implements Issue #571 .
|
| |
|
|
|
|
| |
- sheet showing the first 4kB of pasteboard content and the SQL/CSV accessory view
- pasteboard content will be saved as temp file and read by using the current table/connection encoding
|
| |
|
|
| |
SPConstants.h/m.
|
| |
|
|
| |
- calculate correctly whether global variables are used or not to avoid a crash if the number of columns in the csv file differs
|
| |
|
|
|
|
|
| |
- added UPDATE method which allows to configure an import statement à la:
UPDATE t1 SET col1=foo1, col2=foo2 WHERE col3=baz1 AND col4=baz2
- Insert remaining rows (which are not matched) isn't implemented yet
- note: further tests are needed to ensure it works as expected
|
| |
|
|
| |
- further work on UPDATE (not yet activated)
|
| |
|
|
|
| |
- added Advanced sheet for setting IGNORE, DELAYED, ON DUPLICATE KEY UPDATE '<string>' for INSERT/REPLACE INTO
• if during import CSV Import an error occurred show SP Console
|
| |
|
|
|
| |
- allow to define a list of global variables (incl. a true NULL value)
- such a global variable will be inserted into the mapped table target field for all rows imported
|
| |
|
|
|
|
|
|
| |
- introduced new method:
- (void)showErrorSheetWithMessage:(NSString*)message
to unify it and made all error sheets doc-modal (not app-modal)
- Close button of the errorSheet now also listens at ESC
|
| |
|
|
|
|
| |
- added button "Go Back" ⌥← to go back to the file chooser (mainly for correcting delimiter etc.)
- improved selecting csv file columns for the case that column was set as 'Do not import'; now if set to 'Do not import' the value is hidden, click the user on it the popup menu appears and the operator is set to 'Do import' automatically
• TableDump now remembers the last chosen csv file name
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- changed the way to choose whether a source field should be imported or not by introducing a new table column 'operators'
- clicking at the 'operator's header toggles all operators to 'Import' or 'Do not import'
- added tooltips for each table cell; if file's first line are the headers show them in the tooltips as well
- added checkbox "First line contains fields names" since it'll be clear in this pane whether a file has a header line or not (will be sync with prefs)
- added the possibility to choose the import method: INSERT INTO or REPLACE INTO
• deleted all old field mapper stuff from TableDump and DBView.xib
Notes:
- tests are needed to be sure that this change does not cause mismatches while importing
- symbols for Do (not) import are tendative - maybe use images
- a further import method UPDATE plus an operator '=' will be added soon
- chance to add a new global source variable will come soon
- displaying of source field types will come soon
- semi-automatically matching of source field names and header names will come soon
- the GUI needs some improvements afterwards
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
functions (off by default). Remove the forked "...Sql..." functions, as they're now duplicates, and switch CustomQuery to using the original methods.
- TableDump imports can now process DELIMITERs correctly as a result.
- Alter the TableDump display of tables etc to use TablesList as the source of information, and used cached lists where appropriate for a small speedup. Also means we gain consistent sorting.
- Display procedures and functions in the toggleable list when exporting as SQL
- Tweak the procedure and function export to only export selected items, and also to respect the "export drop syntax" and "export create syntax" checkboxes
- Fix a crash when removing items from the TablesList resulted in an errorneous selection by deselecting all rows before deleting (and preemptively applying the same fix to TableContent)
|
| |
|
|
|
|
|
| |
• if an error occurred while retrieving column or index data in Structure pane reset Structure pane to a stable status, display the error message, and reload Tables List table due to the fact the it's very likely that SP tries to retrieve data from a table which doesn't exist anymore
• fixed spelling of "occurred"
Note: NOT YET DONE: if in Structure view the actual underlying table was deleted or renamed by an other mysql event and the user tries to add/change a field do suppress this attempt safely
|
| |
|
|
|
|
|
| |
release builds, including a large number of 64bit compatibility upgrades and tweaks
- Upgrade RegexKitLite to 3.3
|
| |
|
|
|
|
|
| |
- Exports are now threaded, allowing use of other windows during the export, and can be cancelled (which deletes the partially written export file) including use of the new query cancellation support.
- Correct the text of XML exports and improve progress feedback for XML exports.
- Fix .dot export of tables including foreign keys to use the new foreign key formats.
|
| |
|
|
|
|
|
|
|
|
| |
tied to table list selection. Much faster selection of table to import into if, eg, content view is selected. Fixes weird crashes.
- No longer display views as import targets
- When one import has been cancelled, still allow new imports
- Improve error reporting
- No longer re-sort table/view/etc names returned by TablesList as preferred order is being used for display and the default compare: reverts this
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- Replace the CSV parsing function (arrayForCSV:) with a new SPCSVParser class
- Make speed improvements to SPCSVParser to achieve 1.9x faster parsing than the old arrayForCSV: function
- Rewrite CSV imports to be performed as a streaming import, keeping memory usage much much lower
- CSV field mapping preview is now shown very early on in the import process, as soon as the first hundred rwos are available for a preview
- Progress bars are more consistent and accurate
- CSV rows are grouped into batches of up to 50 (depending on line length) for import, falling back to one-query-per-row if errors occur. The current error reporting level is therefore maintained, but imports of non-erroring data are much much faster.
- Improve processing speed slightly
- Fix some odd edge cases in CSV parsing
This addresses issue #389.
|
| |
|
|
| |
for confirmation, furthermore if a connection is available the user can choose 'Import' instead of loading it into the Query Editor for cases that an user invoked 'Open…' accidentally instead of 'Import…'
|
| |
|
|
|
|
|
|
| |
- Rework CSV export to stream data, significantly reducing memory consumption and so increasing speed and stability when exporting large tables. By default safe/fast streaming is used, but a checkbox is available to select "low memory mode" full streaming, allowing export of any size table in theory. This addresses Issue #224.
- Rework XML export to stream data in the same way, also significantly reducing memory usage and providing the option of using low memory mode.
- Make SQL, CSV and XML export progress bars update more smoothly
- When exporting the current browse view or custom query result, show an indeterminate progress bar when copying large resultsets to avoid the app appearing to hang
|
| |
|
|
|
|
|
| |
- SQL import now reads and processes files in full streaming mode, running queries as they are encountered
- Memory usage during import is significantly reduced, and should stay within a few megabytes; the significant memory use remaining is for query logging
- The progress bar more accurately represents progress and is shown at once (this addresses Issue #320)
|
| |
|
|
|
|
|
| |
download all results as fast as possible from the server, to avoid blocking, but do so in a background thread to allow results processing to start as soon as data is available. Many thanks to Hans-Jörg Bibiko for assistance with this.
- Add an option to the SQL export dialog to allow selection of the full-streaming method, with a warning that it may block table UPDATES/INSERTS.
|
| | |
|
| |
|
|
|
|
|
|
| |
- If there are twenty or more tables, show a table quicksearch/filter at the top of the list, and update the rest of the code to match. This addresses issue #178.
- Select tables and views alphabetically by user's current locale (instead of default MySQL "A B C a b c")
- When adding or duplicating tables, insert them at the correct point
- Fix a number of minor display bugs caused by incorrect interaction with the tables list caches
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Includes all custom code from subclasses CMMCPConnection and CMMCPResult, meaning they have subsequently been removed from the project.
- All previous Sequel Pro specific code in the above subclasses has been removed in favour of the delegate (currently set to TableDocumet) informing the framework of such information.
- All references to CMMCPConnection and CMMCPResult have subsequently been changed to MCPConnection and MCPResult.
- Framework includes MySQL 5.1.36 client libraries and source headers.
- Framework is now built as a 4-way (32/64 bit, i386/PPC arch) binary.
- All import references to <MCPKit_bundled/MCPKit_bundled.h> have been changed to <MCPKit/MCPKit.h>.
- New script 'build-mysql-client.sh' can be used to build the MySQL client libraries from the MySQL source. See the script's header for a list of available options or run it with no arguments to display it's usage.
Note that there are still a few changes to be made to the framework with regard to removing Sequel Pro specific calls to the delegate. These however can be made later on as they have no effect on functionality and are merely design changes.
Also, note that any future development done on the framework should be made to be as 'generic' as possible, with no Sequel Pro specific references. This should allow the framework to be integrated into another project without the need for SP specific code.
|
| |
|
|
|
|
|
|
|
|
| |
be closed when the window is closed, freeing the document's memory
- Update a number of dealloc methods to include more retained memory, and to avoid releasing non-retained memory
- Remove notification observers and delegates where appropriate to avoid issues after document closing
- Fix a couple of memory leaks
- Support window cascading for all windows past the first, using the first window as the autosave window
|
| |
|
|
|
| |
- Fix mutliple-table CSV and XML export when a view is selected - data for the view is now correctly exported
|
| |
|
|
| |
tables, resolving Issue #313
|
| | |
|
| | |
|
| |
|
|
| |
- committing Source
|
| |
|
|
| |
enclosure, escape and line ending characters from the "export multiple tables" dialog instead.
|
| |
|
|
| |
and automatically change the format dropdown to match
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Significantly reduce the queries that have to be performed, improving lag - especially over slow connections (Issue #118; see new controller info under headline code changes).
- Fix Issue #117 properly (export numeric quoting - we now have access to column types and so can quote appropriately).
- Fix Issue #145 (loss of unsigned/null/default attributes when reordering columns).
- Fixes Issue #90 (support for filtering DECIMAL column types)
- Improve table scrolling speed when the table contains long items. (Added a NSFormatter to automatically truncate strings > 150 chars for display purposes only)
- Improved SQL compatibility - for example /* C style comments */ are now correctly ignored in imports and custom queries.
- Add text and symbols emphasising that the table info pane / status view row count is an approximation (partially addresses Issue #141)
- Fixes a major memory leak whenever opening or scrolling tables containing text/blob data.
- SQL import is now faster (SQL parsing part is 3x faster).
- Speed up SQL export (1.5x faster for numeric data; 1.1x faster for string data) and slightly speed up CSV export (~1.1x faster).
- Display sizes on the status view using the byte size formatter, as per table info pane.
Headline code changes:
- Add a new NSMutableString subclass, SPSQLParser. See the header file for documentation and overview, but in short it's a centralised place for SQL parsing. Centralises and improves parsing, improves comment support, improves quoting support. Despite the improved featureset this is also faster than the previous distributed implementations - for example, when used to replace the old splitQueries:, > 3x speedup.
- Implement a new controller which handles a structure and status cache for the current table, and provides structure parsing for specified tables. This cache is now used throughout the code, reducing the queries that have to be performed and providing additional information about the table structure for use; I think it also improves column type format slightly.
- The table info pane and the status view now draw all their data from the cache.
Tweaks:
- Table encoding is now detected directly instead of being derived from the collation - increased accuracy and cope with the DEFAULT encoding.
- Comments and formatting cleaned up in bits I was working on, obviously.
- A couple of methods - particularly [tablesListInstance table] and [tableDocument encoding] - have been renamed to avoid conflicts and fix code warnings.
Future improvements now possible:
- As we now have access to column types and other information, we can provide per-type behaviour where desired.
- The table parsing doesn't currently pull out comments or table indices, together with one or two other attributes. Some of this would be useful for display; some, such as indices, could be used to draw the table structure view as long as we're happy discarding a couple of columns (ie cardinality!)
|
| |
|
|
| |
import panel
|
| |
|
|
| |
importing on a separate thread.
|
|
|
structure.
|
