Core Changelog for Super Speedy Imports

Contributors: dhilditch
Donate link: https://www.superspeedyplugins.com/
Tags: imports, speed, performance, fast
Requires at least: 6.0
Tested up to: 6.4
Stable tag: This is alpha
Requires PHP: 7.0 License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html

Description

Import 1 million items in 100 minutes.

Known Issues

There are many known issues! 1. If you import a config from elsewhere, but you do not have the taxonomies configured, they will all default to the first taxonomy - a taxonomy expor/import tool has been added 2. You MUST have an SKU or it will not work at all

Coming Soon

  • Use friendly names from template for postmeta
  • Add default post template
  • Hide hidden fields
  • Show attached CSV file for import
  • Add option for an import to be executed against a different CSV file

Changelog

1.48 (20th December 2024)

  • Added version number to output so I can be sure which version clients are using - some hosts set opcache really high so it can still be using old code on some hosts even if updated through wp-admin

1.47 (20th December 2024)

  • Fixed bug with hierarchies being too long - previous limit of 200 characters was lifted (no limit now other than 200 char limit for each term name in the hierarchy)
    • However, this is in contradiction with WordPress - because they uniquify the slug, their uniquify function can create slugs > 200 characters which then fail
    • There's a new filter added to wp_unique_term_slug to reduce the size of the slug and still ensure it's unique so our import can handle any number of X > Y > Z terms so long as each component term name is < 200 characters

1.46 (18th December 2024)

  • Fixed memory leak in fix-attributes stage - it was from a call to WordPress core function update_user_meta - even though I only asked to update user meta, it also caches the meta for that item

1.45 (13th December 2024)

  • Added PHP directive to display errors to CLI output for cases of some hosting environments having everything off
  • Added better debug to failed function eval so you now find out exactly which column has a bad function defined for it

1.44 (12th December 2024)

  • Fixed issue with saving PHP functions - was working if the function was still in codemirror/expanded view, otherwise was saving nothing

1.43 (11th December 2024)

  • Fixed taxonomy matching bug introduced in 1.35
  • Fixed save import broken if taxonomy with function (not csv mapping) was in the import
  • Found and fixed a bug affecting terms with parents being mistakenly matched with parentless terms
    • this in turn caused some terms to not be attempted to import and subsequent checks to fail because of these terms being missing
  • Added CSV file name for configured template to template info at top of interface
  • Added links to let admins see products affected by this import
    • also added dropdown filter to edit.php?post_type=product so users can filter on there to view products last affected by X import
  • Big speed boost for upsert-relationships stage - it's as fast as it should be again, e.g. 50,000 items 0.5s now compared to 5+ minutes previously
  • Strengthed wp_posts update so it runs one update per column and checks which rows to set based on source being not null and not empty
  • Added performance info for load-csv stage which is often the slowest stage

1.42 (6th December 2024)

  • Proper fix for tab-separated files - previously I only fixed the import interface, now fixed the actual import run too to handle tab separated files properly

1.41 (6th December 2024)

  • Fix for tab-separated files

1.40 (29th November 2024)

  • Fix to allow spaces in CSV file names...

1.39 (29th November 2024)

  • Added alert if trying to load an import to edit when the CSV file has been deleted - it's currently needed for reference
  • Added early-access auto-update prevention mechanism - users have to very clearly and deliberately enable auto-updates to mess up their live site - if you want to enable auto updates on dev or staging, add define('SSI_ALLOW_AUTO_UPDATE', true); to your wp-config.php
  • Added run now button to interface for small imports

1.38 (29th November 2024)

  • Improved default template to remove unused fields
  • Hidden tax and postmeta entries which are meant to be hidden
  • Added prevent creation of import with no name
  • Added duplicate import name check
  • Added new cli parameter list-files - e.g. wp ssi 20 list-files - will list files compatible with this import template
  • Added new cli --file parameter - e.g. wp ssi 20 --file="alternative.csv" will use import 20 but against the alternative.csv file - this will barf if the headers are different

1.37 (27th November 2024)

  • Fix for new settings to clear the local transient cache when the license key is changed
  • Added a new 'recheck now' button on the license key page for when users renew or upgrade

1.36 (21st November 2024)

  • Tidy up and standardisation of new license/settings page

1.35 (21st November 2024)

  • Speed boost for stage 1 - load-csv stage
  • Fix for stage 2 - import-taxonomies - no more reporting error when no error occurred, and fixed bug with duplicate term names in different taxonomies
  • Upgraded warning messages inside functions to Fatal Errors - this helps debug failing functions, just fix all the warnings so the code can run

1.34 (20th November 2024)

  • Fixed broken function calls - since I added the pretty textareas, the functions were not saving properly
  • Updated the JS to use time() for admin so ugosprint definitely gets the latest JS

1.33 (19th November 2024)

  • Added _default_attributes hidden field to batch import table
  • Use _default_attributes to store first variation from table selected per variable product
  • Updates variable product parents efficiently to set the default variation using _default_attributes
  • Fixed all slugs to be sanitized - for post_name and for attribute names when they are used as keys

1.32 (16th November 2024)

  • Fixed JS/CSS loading issue for certain versions of Scalability Pro (shared Settings file)

1.31 (13th November 2024)

  • Fixed issue with additional options (delete items missing from previous import) from appearing multiple times as you loaded each import
  • Added code to hide the 'hidden' fields in posts

1.30 (12th November 2024)

  • Fixed various warnings

1.29 (12th November 2024)

  • Fixed bug where everything was being marked as variable product, even simple products
  • Fixed new bug with postmeta _regular_price where it was checking the function rather than the stage table for whether it should upsert or not

1.28 (12th November 2024)

  • update-postmeta stage adjusted to increase performance
  • Updated post-update and post-insert to not set the value if the source column is null
  • Added post-slug field - by default, a sanitized post_title will be used for the post slug, but you can overrule this by setting a value for the post slug - if slug and post title are not set, the post_name will not be changed
  • Added variation control into the taxonomy section
  • Integrated variation control with previous hard-coded variations list

1.27 (8th November 2024)

  • Reduced key size for hierarchical tables to acommodate 4-byte unicode collations
  • Removed fallback to try to convert from wpallimport function - it was failing badly

1.26 (8th November 2024)

  • Renamed stages to useful names
  • Tidied up SQL code especially since no guarantee of execution order
  • Fixed some warnings

1.25 (8th November 2024)

  • Commented out the eval method for function calling in postmeta

1.24 (8th November 2024)

  • Added variable product capability

    • in dev, you need to configure your variation attributes with:

      define('SSI_HARD_CODED_VARIABLE_TAXONOMY_ARRAY', array('pa_size', 'pa_colour'));

1.23 (7th November 2024)

  • Added new code to let templates add and alter additional options
  • Added additional options section to end of import with save/load/conditional appearance
  • Added 3 new delete options to let people configure how they want their deletes handled

1.22 (4th November 2024)

  • Fixed bug with flat term import where import failed if term name was empty in csv

1.21 (22nd October 2024)

  • Fixed hierarchical taxonomies so they are correctly set with parents in the database - also confirmed existing terms are being retrieved correctly
  • Also upgraded multi-term inserts, so if you have e.g. Cat1|Cat2|Cat3 > Cat4 then the item will be allocated to Cat 1, Cat 2 and Cat 4 correctly
  • Fixed term inserts and checks for terms when you have the likes of: Salt & Pepper > Salt|Salt & Pepper > Pepper, so it correctly assigns the parent for the 2nd term Pepper to be Salt & Pepper
  • Updated code to use the decimal separator correctly for prices

1.20 (3rd October 2024)

  • Fixed bug in wp ssi 3 save_posts command - it was just re-running all instead of doing the save_posts

1.19 (30th September 2024)

  • Fixed bug for terms with html encodable text in their name (typically &)
  • Improved speed by fixing collation issue
    • rather than trusting default collations in WP, i pull the collation from wp_terms since this is what ultimately is joined to on text columns
  • Hard errors now correctly stop the script and output the error in RED
  • Vastly improved debug and error output to help trace problems more quickly

1.18 (25th September 2024)

  • Added gallery image support
  • Added checks at start to ensure you have an SKU (for now, there will be other ways of identifying the same item in future)

1.17 (24th September 2024)

  • Added featured image import
  • Added --verbose mode

1.16 (17th September 2024)

  • Fixed the collation issue some users were having
  • Added taxonomy export/import as extra option - this makes it far easier for me to replicate exactly what users see
  • Fixed an issue with importing terms where the term name was used in another taxonomy
  • Tested multiple imports, everything imported fully

1.15 (14th September 2024)

  • Added pa_ real name indicators to interface for taxonomy
  • Fixed flat taxonomy issue with missing terms after import - all terms are fully loaded now

1.14 (12th September 2024)

  • Improved error logging for broken functions so you can see which one needs attention

1.13 (6th September 2024)

  • Added optional collation config for those who have mismatching collations in their defaults versus actual tables
    • add the following to your wp-config: define('SSI_COLLATION', 'DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci');
    • change the collation to whatever your tables are and this will fix collation issues
    • to see your table collations run SHOW TABLE STATUS

1.12 (4th September 2024)

  • Fixed stupid mistake of hardcoding my dev table name into stage 2!

1.11 (4th September 2024)

  • Much more work and fixes for hiearchical taxonomies
    • tested with flat and hiearchical, tested 5 levels deep
    • only setting leaf term, not all the ancestors
  • Fixed attributes
    • product attributes now work properly and are loaded in stage 10
    • this stage may be removed if I can get this back into stage 1 where the array should be defined and where this should work

1.10 (3rd September 2024)

  • Speed boost by eliminating duplicates in the _hieararchies table

1.09 (3rd September 2024)

  • Code inspected everything related to taxonomies
    • fixed hierarchical taxonomies not being set in destination

1.08 (29th August 2024)

  • Added debug for user functions
  • Added check for array type to avoid potential fatal error

1.07 (28th August 2024)

  • Added export/import for templates
    • export will export to file, but also gives you the export in the textarea to copy/paste
    • import involves pasting the import config in the textarea then hitting import
  • Fixed error with decimal place not being sent to the server for new imports
  • Changed default mapping types to 'Direct Mapping' rather than 'function' since this should be most of the cases
  • Ajaxified everything to reduce confusion for new users

1.06 (13th June 2024)

  • Added txt file compatibility
  • Added semi-colon separator compatibility
  • Tidied up the interface a little
  • Added multi-hierachical term capability (so you can have 2+ product cats per item)
  • Fixed other taxonomy related bugs
  • Optimised the code a bit
  • Found a bunch of bugs in taxonomy, fixed more
  • Added multi-product_cat compatibility
  • Fixed collation and charset so it pulls from $wpdb

1.05 (31st January 2024)

  • Fixed additional columns defined in template
    • these are now correctly added to the flat mappings table if the user did not add them
    • the mapData function is properly called now so any template mappings or additional data massage can occur

1.04 (30th January 2024)

  • Added commas as decimals handler in the Woo import template
  • Added infinite time limit in case wp cli badly configured

1.03 (30th January 2024)

  • Fixed import bug to handle dashes in attribute names and postmeta keys

1.02 (30th January 2024)

  • Added functions editor to interface
    • also eval these functions at import time through wp cli
    • you must trust people who can edit these functions obviously

1.01 (29th January 2024)

  • JS Fix for new imports to load default values for postmeta

1.00 (29th January 2024)

  • Very first working version including interface
    • please read the knowledge-base guide to understand how to use this, in particular how to use the function system
Super Speedy Plugins
Logo