Core Changelog for Super Speedy Imports
Contributors: dhilditch
Donate link: https://www.superspeedyplugins.com/
Tags: imports, speed, performance, fast
Requires at least: 6.0
Tested up to: 6.4
Stable tag: This is alpha
Requires PHP: 7.0
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html
Description
Import 1 million items in 100 minutes.
Known Issues
There are many known issues! 1. If you import a config from elsewhere, but you do not have the taxonomies configured, they will all default to the first taxonomy - a taxonomy expor/import tool has been added 2. You MUST have an SKU or it will not work at all
Coming Soon
- Use friendly names from template for postmeta
- Add default post template
- Hide hidden fields
- Show attached CSV file for import
- Add option for an import to be executed against a different CSV file
Changelog
1.48 (20th December 2024)
- Added version number to output so I can be sure which version clients are using - some hosts set opcache really high so it can still be using old code on some hosts even if updated through wp-admin
1.47 (20th December 2024)
- Fixed bug with hierarchies being too long - previous limit of 200 characters was lifted (no limit now other than 200 char limit for each term name in the hierarchy)
- However, this is in contradiction with WordPress - because they uniquify the slug, their uniquify function can create slugs > 200 characters which then fail
- There's a new filter added to wp_unique_term_slug to reduce the size of the slug and still ensure it's unique so our import can handle any number of X > Y > Z terms so long as each component term name is < 200 characters
1.46 (18th December 2024)
- Fixed memory leak in fix-attributes stage - it was from a call to WordPress core function update_user_meta - even though I only asked to update user meta, it also caches the meta for that item
1.45 (13th December 2024)
- Added PHP directive to display errors to CLI output for cases of some hosting environments having everything off
- Added better debug to failed function eval so you now find out exactly which column has a bad function defined for it
1.44 (12th December 2024)
- Fixed issue with saving PHP functions - was working if the function was still in codemirror/expanded view, otherwise was saving nothing
1.43 (11th December 2024)
- Fixed taxonomy matching bug introduced in 1.35
- Fixed save import broken if taxonomy with function (not csv mapping) was in the import
- Found and fixed a bug affecting terms with parents being mistakenly matched with parentless terms
- this in turn caused some terms to not be attempted to import and subsequent checks to fail because of these terms being missing
- Added CSV file name for configured template to template info at top of interface
- Added links to let admins see products affected by this import
- also added dropdown filter to edit.php?post_type=product so users can filter on there to view products last affected by X import
- Big speed boost for upsert-relationships stage - it's as fast as it should be again, e.g. 50,000 items 0.5s now compared to 5+ minutes previously
- Strengthed wp_posts update so it runs one update per column and checks which rows to set based on source being not null and not empty
- Added performance info for load-csv stage which is often the slowest stage
1.42 (6th December 2024)
- Proper fix for tab-separated files - previously I only fixed the import interface, now fixed the actual import run too to handle tab separated files properly
1.41 (6th December 2024)
- Fix for tab-separated files
1.40 (29th November 2024)
- Fix to allow spaces in CSV file names...
1.39 (29th November 2024)
- Added alert if trying to load an import to edit when the CSV file has been deleted - it's currently needed for reference
- Added early-access auto-update prevention mechanism - users have to very clearly and deliberately enable auto-updates to mess up their live site - if you want to enable auto updates on dev or staging, add define('SSI_ALLOW_AUTO_UPDATE', true); to your wp-config.php
- Added run now button to interface for small imports
1.38 (29th November 2024)
- Improved default template to remove unused fields
- Hidden tax and postmeta entries which are meant to be hidden
- Added prevent creation of import with no name
- Added duplicate import name check
- Added new cli parameter list-files - e.g. wp ssi 20 list-files - will list files compatible with this import template
- Added new cli --file parameter - e.g. wp ssi 20 --file="alternative.csv" will use import 20 but against the alternative.csv file - this will barf if the headers are different
1.37 (27th November 2024)
- Fix for new settings to clear the local transient cache when the license key is changed
- Added a new 'recheck now' button on the license key page for when users renew or upgrade
1.36 (21st November 2024)
- Tidy up and standardisation of new license/settings page
1.35 (21st November 2024)
- Speed boost for stage 1 - load-csv stage
- Fix for stage 2 - import-taxonomies - no more reporting error when no error occurred, and fixed bug with duplicate term names in different taxonomies
- Upgraded warning messages inside functions to Fatal Errors - this helps debug failing functions, just fix all the warnings so the code can run
1.34 (20th November 2024)
- Fixed broken function calls - since I added the pretty textareas, the functions were not saving properly
- Updated the JS to use time() for admin so ugosprint definitely gets the latest JS
1.33 (19th November 2024)
- Added _default_attributes hidden field to batch import table
- Use _default_attributes to store first variation from table selected per variable product
- Updates variable product parents efficiently to set the default variation using _default_attributes
- Fixed all slugs to be sanitized - for post_name and for attribute names when they are used as keys
1.32 (16th November 2024)
- Fixed JS/CSS loading issue for certain versions of Scalability Pro (shared Settings file)
1.31 (13th November 2024)
- Fixed issue with additional options (delete items missing from previous import) from appearing multiple times as you loaded each import
- Added code to hide the 'hidden' fields in posts
1.30 (12th November 2024)
- Fixed various warnings
1.29 (12th November 2024)
- Fixed bug where everything was being marked as variable product, even simple products
- Fixed new bug with postmeta _regular_price where it was checking the function rather than the stage table for whether it should upsert or not
1.28 (12th November 2024)
- update-postmeta stage adjusted to increase performance
- Updated post-update and post-insert to not set the value if the source column is null
- Added post-slug field - by default, a sanitized post_title will be used for the post slug, but you can overrule this by setting a value for the post slug - if slug and post title are not set, the post_name will not be changed
- Added variation control into the taxonomy section
- Integrated variation control with previous hard-coded variations list
1.27 (8th November 2024)
- Reduced key size for hierarchical tables to acommodate 4-byte unicode collations
- Removed fallback to try to convert from wpallimport function - it was failing badly
1.26 (8th November 2024)
- Renamed stages to useful names
- Tidied up SQL code especially since no guarantee of execution order
- Fixed some warnings
1.25 (8th November 2024)
- Commented out the eval method for function calling in postmeta
1.24 (8th November 2024)
Added variable product capability
in dev, you need to configure your variation attributes with:
define('SSI_HARD_CODED_VARIABLE_TAXONOMY_ARRAY', array('pa_size', 'pa_colour'));
1.23 (7th November 2024)
- Added new code to let templates add and alter additional options
- Added additional options section to end of import with save/load/conditional appearance
- Added 3 new delete options to let people configure how they want their deletes handled
1.22 (4th November 2024)
- Fixed bug with flat term import where import failed if term name was empty in csv
1.21 (22nd October 2024)
- Fixed hierarchical taxonomies so they are correctly set with parents in the database - also confirmed existing terms are being retrieved correctly
- Also upgraded multi-term inserts, so if you have e.g. Cat1|Cat2|Cat3 > Cat4 then the item will be allocated to Cat 1, Cat 2 and Cat 4 correctly
- Fixed term inserts and checks for terms when you have the likes of: Salt & Pepper > Salt|Salt & Pepper > Pepper, so it correctly assigns the parent for the 2nd term Pepper to be Salt & Pepper
- Updated code to use the decimal separator correctly for prices
1.20 (3rd October 2024)
- Fixed bug in wp ssi 3 save_posts command - it was just re-running all instead of doing the save_posts
1.19 (30th September 2024)
- Fixed bug for terms with html encodable text in their name (typically &)
- Improved speed by fixing collation issue
- rather than trusting default collations in WP, i pull the collation from wp_terms since this is what ultimately is joined to on text columns
- Hard errors now correctly stop the script and output the error in RED
- Vastly improved debug and error output to help trace problems more quickly
1.18 (25th September 2024)
- Added gallery image support
- Added checks at start to ensure you have an SKU (for now, there will be other ways of identifying the same item in future)
1.17 (24th September 2024)
- Added featured image import
- Added --verbose mode
1.16 (17th September 2024)
- Fixed the collation issue some users were having
- Added taxonomy export/import as extra option - this makes it far easier for me to replicate exactly what users see
- Fixed an issue with importing terms where the term name was used in another taxonomy
- Tested multiple imports, everything imported fully
1.15 (14th September 2024)
- Added pa_ real name indicators to interface for taxonomy
- Fixed flat taxonomy issue with missing terms after import - all terms are fully loaded now
1.14 (12th September 2024)
- Improved error logging for broken functions so you can see which one needs attention
1.13 (6th September 2024)
- Added optional collation config for those who have mismatching collations in their defaults versus actual tables
- add the following to your wp-config: define('SSI_COLLATION', 'DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci');
- change the collation to whatever your tables are and this will fix collation issues
- to see your table collations run SHOW TABLE STATUS
1.12 (4th September 2024)
- Fixed stupid mistake of hardcoding my dev table name into stage 2!
1.11 (4th September 2024)
- Much more work and fixes for hiearchical taxonomies
- tested with flat and hiearchical, tested 5 levels deep
- only setting leaf term, not all the ancestors
- Fixed attributes
- product attributes now work properly and are loaded in stage 10
- this stage may be removed if I can get this back into stage 1 where the array should be defined and where this should work
1.10 (3rd September 2024)
- Speed boost by eliminating duplicates in the _hieararchies table
1.09 (3rd September 2024)
- Code inspected everything related to taxonomies
- fixed hierarchical taxonomies not being set in destination
1.08 (29th August 2024)
- Added debug for user functions
- Added check for array type to avoid potential fatal error
1.07 (28th August 2024)
- Added export/import for templates
- export will export to file, but also gives you the export in the textarea to copy/paste
- import involves pasting the import config in the textarea then hitting import
- Fixed error with decimal place not being sent to the server for new imports
- Changed default mapping types to 'Direct Mapping' rather than 'function' since this should be most of the cases
- Ajaxified everything to reduce confusion for new users
1.06 (13th June 2024)
- Added txt file compatibility
- Added semi-colon separator compatibility
- Tidied up the interface a little
- Added multi-hierachical term capability (so you can have 2+ product cats per item)
- Fixed other taxonomy related bugs
- Optimised the code a bit
- Found a bunch of bugs in taxonomy, fixed more
- Added multi-product_cat compatibility
- Fixed collation and charset so it pulls from $wpdb
1.05 (31st January 2024)
- Fixed additional columns defined in template
- these are now correctly added to the flat mappings table if the user did not add them
- the mapData function is properly called now so any template mappings or additional data massage can occur
1.04 (30th January 2024)
- Added commas as decimals handler in the Woo import template
- Added infinite time limit in case wp cli badly configured
1.03 (30th January 2024)
- Fixed import bug to handle dashes in attribute names and postmeta keys
1.02 (30th January 2024)
- Added functions editor to interface
- also eval these functions at import time through wp cli
- you must trust people who can edit these functions obviously
1.01 (29th January 2024)
- JS Fix for new imports to load default values for postmeta
1.00 (29th January 2024)
- Very first working version including interface
- please read the knowledge-base guide to understand how to use this, in particular how to use the function system