Changelog

1.62 (27th March 2025)

• Fixed error handling so that if an actual SQL error occurs then it aborts properly
• Upgraded hard-error reporting so that it uses the :error from WP CLI (it shows in red in the console rather than white)
• Upgraded insert-postmeta stage so that it will skip missing meta columns from the import table which can happen, e.g. if _sale_price was not defined
• Upgraded valid-CSV detection so it works for exports from other places - instead of checking mime type, actually reads the first header row

1.61 (25th March 2025)

• Fixed bug with deletes from refactoring job

1.60 (25th March 2025)

• Added all remaining missing columns for config from wp_posts
  • defaults will be used if they are missing (e.g. author will be the current user, date will be current date)
  • any items NOT configured will guarantee those items will NOT be updated, but may be inserted as required (e.g. author, dates, other sensible defaults)
• Fixed last refactor job - it was working for posts, but not for products
• Added new CPT template in case anyone wants to experiment with post types other than post or product
• Found and fixed 1 place preventing simultaneous import execution (so long as it's different import ids)
• Found and fixed a bunch of other bugs that arose from the refactor job
• In particular, inserts always insert defaults now for any unspecified import whereas updates never will use defaults

1.59 (10th March 2025)

• Minor refactoring
• Added Post template, tested, working
• Separated all product related stuff out into the WooCommerce-specific template
• Added import history to admin interface
• Added import details to admin interface
• Hardened checks for additional options
• Fixed issue with HTML special characters in term names (flat terms & hierarchical)
• Upgraded the sanitize term function to exactly mimic wp_insert_term - should fix any remaining broken term names
• Added new import-taxonomies stage specific to posts and specific to products
• Added fallback for terms failing to insert to grab the duplicate term_id and continue the import
• Added sanitize_terms to clean up the term mess I made
• Added 2 new tables to record ambiguous term names - e.g. if Dave's stuff is a term name, that could be inserted either html encoded or not, and may not be discovered by the join
• Added handler to update the term_taxonomy_id's from the ambiguous terms tables
• Added SSI_RUNNING_IMPORT definition in case anything wants to do or not do stuff when SSI is running
• Batched up the insert-posts command into 100,000 per batch and added performance metrics to trace slow (200 minutes) performance for 2.7 million row CSV file
• Renamed save_posts to save-posts, renamed sanitize_terms to sanitize-terms
• Added --perf option to save_posts to show what's taking time, which function, which plugin

1.58 (4th March 2025)

• Code refactoring of runtime
• Improved performance reporting summary
• Improved logging
• Fixed issue with attributes to ensure it only updates attributes mentioned in the import
• Improved resilience of save_posts for the Woo Product template, now uses wc_get_product and $product->save etc
• Added extra output to end of import to inform admin of how to run the builds for regular woocommerce attributes
• Fixed issue with attribute/taxonomy import - changed to use wc_create_attribute instead of standard wordpress taxonomy code for attributes

1.57 (21st February 2025)

• Fixed force delete not working
• Fixed 'keep sold' items option not being recognised
• Added lots more logging everywhere
• Added import summary table at end of import

1.56 (7th February 2025)

• Fixed progress indicator on save_posts

1.55 (5th February 2025)

• Fixed 'permanently delete' items vs move to trash - thanks to @AdamDar for this one!
• Fixed issue with SQL to keep 'previously sold items'

1.54 (4th February 2025)

• Improved RAM usage for fix-attributes stage
• Added new execution URL which can be useful for profiling with Code Profiler Pro

1.53 (14th January 2025)

• Further improvements to image download to ensure it works on as many hosts as possible
• Fixed fatal error in some paths for image downloads

1.52 (14th January 2025)

• Increased image download timeout
• Fixed issue with source image URLs containing query parameters

1.51 (10th January 2025)

• Fixed CSV upload ajax for brand new users so that the CSV file correctly appears in the dropdown after upload
• Fixed New Import creation for brand new imports, so that the import loads properly after creation
• Fixed deletes - it was referencing the old meta key name, updated to use ssi_updated and deletes are working like before

1.50 (24th December 2024)

• Added continue-on-error functionality for CSV imports - if a row or term fails, it will be logged and the import will continue
• Fixed function editor initialization to properly use full code editor with PHP syntax highlighting

1.49 (23rd December 2024)

• Optimised load-csv stage by eliminating need for get_existing_terms at small cost of extra RAM consumption
• Small optimisation for the insert_flat_terms code by moving index creation on that table to the end of load-csv stage
• Optimised load-csv stage using more memory efficient way of reading the CSV file

1.48 (20th December 2024)

• Added version number to output so I can be sure which version clients are using - some hosts set opcache really high so it can still be using old code on some hosts even if updated through wp-admin

1.47 (20th December 2024)

• Fixed bug with hierarchies being too long - previous limit of 200 characters was lifted (no limit now other than 200 char limit for each term name in the hierarchy)
  • However, this is in contradiction with WordPress - because they uniquify the slug, their uniquify function can create slugs > 200 characters which then fail
  • There's a new filter added to wp_unique_term_slug to reduce the size of the slug and still ensure it's unique so our import can handle any number of X > Y > Z terms so long as each component term name is < 200 characters

1.46 (18th December 2024)

• Fixed memory leak in fix-attributes stage - it was from a call to WordPress core function update_user_meta - even though I only asked to update user meta, it also caches the meta for that item

1.45 (13th December 2024)

• Added PHP directive to display errors to CLI output for cases of some hosting environments having everything off
• Added better debug to failed function eval so you now find out exactly which column has a bad function defined for it

1.44 (12th December 2024)

• Fixed issue with saving PHP functions - was working if the function was still in codemirror/expanded view, otherwise was saving nothing

1.43 (11th December 2024)

• Fixed taxonomy matching bug introduced in 1.35
• Fixed save import broken if taxonomy with function (not csv mapping) was in the import
• Found and fixed a bug affecting terms with parents being mistakenly matched with parentless terms
  • this in turn caused some terms to not be attempted to import and subsequent checks to fail because of these terms being missing
• Added CSV file name for configured template to template info at top of interface
• Added links to let admins see products affected by this import
  • also added dropdown filter to edit.php?post_type=product so users can filter on there to view products last affected by X import
• Big speed boost for upsert-relationships stage - it's as fast as it should be again, e.g. 50,000 items 0.5s now compared to 5+ minutes previously
• Strengthed wp_posts update so it runs one update per column and checks which rows to set based on source being not null and not empty
• Added performance info for load-csv stage which is often the slowest stage

1.42 (6th December 2024)

• Proper fix for tab-separated files - previously I only fixed the import interface, now fixed the actual import run too to handle tab separated files properly

1.41 (6th December 2024)

• Fix for tab-separated files

1.40 (29th November 2024)

• Fix to allow spaces in CSV file names...

1.39 (29th November 2024)

• Added alert if trying to load an import to edit when the CSV file has been deleted - it's currently needed for reference
• Added early-access auto-update prevention mechanism - users have to very clearly and deliberately enable auto-updates to mess up their live site - if you want to enable auto updates on dev or staging, add define('SSI_ALLOW_AUTO_UPDATE', true); to your wp-config.php
• Added run now button to interface for small imports

1.38 (29th November 2024)

• Improved default template to remove unused fields
• Hidden tax and postmeta entries which are meant to be hidden
• Added prevent creation of import with no name
• Added duplicate import name check
• Added new cli parameter list-files - e.g. wp ssi 20 list-files - will list files compatible with this import template
• Added new cli --file parameter - e.g. wp ssi 20 --file="alternative.csv" will use import 20 but against the alternative.csv file - this will barf if the headers are different

1.37 (27th November 2024)

• Fix for new settings to clear the local transient cache when the license key is changed
• Added a new 'recheck now' button on the license key page for when users renew or upgrade

1.36 (21st November 2024)

• Tidy up and standardisation of new license/settings page

1.35 (21st November 2024)

• Speed boost for stage 1 - load-csv stage
• Fix for stage 2 - import-taxonomies - no more reporting error when no error occurred, and fixed bug with duplicate term names in different taxonomies
• Upgraded warning messages inside functions to Fatal Errors - this helps debug failing functions, just fix all the warnings so the code can run

1.34 (20th November 2024)

• Fixed broken function calls - since I added the pretty textareas, the functions were not saving properly
• Updated the JS to use time() for admin so ugosprint definitely gets the latest JS

1.33 (19th November 2024)

• Added _default_attributes hidden field to batch import table
• Use _default_attributes to store first variation from table selected per variable product
• Updates variable product parents efficiently to set the default variation using _default_attributes
• Fixed all slugs to be sanitized - for post_name and for attribute names when they are used as keys

1.32 (16th November 2024)

• Fixed JS/CSS loading issue for certain versions of Scalability Pro (shared Settings file)

1.31 (13th November 2024)

• Fixed issue with additional options (delete items missing from previous import) from appearing multiple times as you loaded each import
• Added code to hide the 'hidden' fields in posts

1.30 (12th November 2024)

• Fixed various warnings

1.29 (12th November 2024)

• Fixed bug where everything was being marked as variable product, even simple products
• Fixed new bug with postmeta _regular_price where it was checking the function rather than the stage table for whether it should upsert or not

1.28 (12th November 2024)

• update-postmeta stage adjusted to increase performance
• Updated post-update and post-insert to not set the value if the source column is null
• Added post-slug field - by default, a sanitized post_title will be used for the post slug, but you can overrule this by setting a value for the post slug - if slug and post title are not set, the post_name will not be changed
• Added variation control into the taxonomy section
• Integrated variation control with previous hard-coded variations list

1.27 (8th November 2024)

• Reduced key size for hierarchical tables to acommodate 4-byte unicode collations
• Removed fallback to try to convert from wpallimport function - it was failing badly

1.26 (8th November 2024)

• Renamed stages to useful names
• Tidied up SQL code especially since no guarantee of execution order
• Fixed some warnings

1.25 (8th November 2024)

• Commented out the eval method for function calling in postmeta

1.24 (8th November 2024)

• Added variable product capability

  • in dev, you need to configure your variation attributes with:

    define('SSI_HARD_CODED_VARIABLE_TAXONOMY_ARRAY', array('pa_size', 'pa_colour'));

1.23 (7th November 2024)

• Added new code to let templates add and alter additional options
• Added additional options section to end of import with save/load/conditional appearance
• Added 3 new delete options to let people configure how they want their deletes handled

1.22 (4th November 2024)

• Fixed bug with flat term import where import failed if term name was empty in csv

1.21 (22nd October 2024)

• Fixed hierarchical taxonomies so they are correctly set with parents in the database - also confirmed existing terms are being retrieved correctly
• Also upgraded multi-term inserts, so if you have e.g. Cat1|Cat2|Cat3 > Cat4 then the item will be allocated to Cat 1, Cat 2 and Cat 4 correctly
• Fixed term inserts and checks for terms when you have the likes of: Salt & Pepper > Salt|Salt & Pepper > Pepper, so it correctly assigns the parent for the 2nd term Pepper to be Salt & Pepper
• Updated code to use the decimal separator correctly for prices

1.20 (3rd October 2024)

• Fixed bug in wp ssi 3 save_posts command - it was just re-running all instead of doing the save_posts

1.19 (30th September 2024)

• Fixed bug for terms with html encodable text in their name (typically &)
• Improved speed by fixing collation issue
  • rather than trusting default collations in WP, i pull the collation from wp_terms since this is what ultimately is joined to on text columns
• Hard errors now correctly stop the script and output the error in RED
• Vastly improved debug and error output to help trace problems more quickly

1.18 (25th September 2024)

• Added gallery image support
• Added checks at start to ensure you have an SKU (for now, there will be other ways of identifying the same item in future)

1.17 (24th September 2024)

• Added featured image import
• Added --verbose mode

1.16 (17th September 2024)

• Fixed the collation issue some users were having
• Added taxonomy export/import as extra option - this makes it far easier for me to replicate exactly what users see
• Fixed an issue with importing terms where the term name was used in another taxonomy
• Tested multiple imports, everything imported fully

1.15 (14th September 2024)

• Added pa_ real name indicators to interface for taxonomy
• Fixed flat taxonomy issue with missing terms after import - all terms are fully loaded now

1.14 (12th September 2024)

• Improved error logging for broken functions so you can see which one needs attention

1.13 (6th September 2024)

• Added optional collation config for those who have mismatching collations in their defaults versus actual tables
  • add the following to your wp-config: define('SSI_COLLATION', 'DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci');
  • change the collation to whatever your tables are and this will fix collation issues
  • to see your table collations run SHOW TABLE STATUS

1.12 (4th September 2024)

• Fixed stupid mistake of hardcoding my dev table name into stage 2!

1.11 (4th September 2024)

• Much more work and fixes for hiearchical taxonomies
  • tested with flat and hiearchical, tested 5 levels deep
  • only setting leaf term, not all the ancestors
• Fixed attributes
  • product attributes now work properly and are loaded in stage 10
  • this stage may be removed if I can get this back into stage 1 where the array should be defined and where this should work

1.10 (3rd September 2024)

• Speed boost by eliminating duplicates in the _hieararchies table

1.09 (3rd September 2024)

• Code inspected everything related to taxonomies
  • fixed hierarchical taxonomies not being set in destination

1.08 (29th August 2024)

• Added debug for user functions
• Added check for array type to avoid potential fatal error

1.07 (28th August 2024)

• Added export/import for templates
  • export will export to file, but also gives you the export in the textarea to copy/paste
  • import involves pasting the import config in the textarea then hitting import
• Fixed error with decimal place not being sent to the server for new imports
• Changed default mapping types to 'Direct Mapping' rather than 'function' since this should be most of the cases
• Ajaxified everything to reduce confusion for new users

1.06 (13th June 2024)

• Added txt file compatibility
• Added semi-colon separator compatibility
• Tidied up the interface a little
• Added multi-hierachical term capability (so you can have 2+ product cats per item)
• Fixed other taxonomy related bugs
• Optimised the code a bit
• Found a bunch of bugs in taxonomy, fixed more
• Added multi-product_cat compatibility
• Fixed collation and charset so it pulls from $wpdb

1.05 (31st January 2024)

• Fixed additional columns defined in template
  • these are now correctly added to the flat mappings table if the user did not add them
  • the mapData function is properly called now so any template mappings or additional data massage can occur

1.04 (30th January 2024)

• Added commas as decimals handler in the Woo import template
• Added infinite time limit in case wp cli badly configured

1.03 (30th January 2024)

• Fixed import bug to handle dashes in attribute names and postmeta keys

1.02 (30th January 2024)

• Added functions editor to interface
  • also eval these functions at import time through wp cli
  • you must trust people who can edit these functions obviously

1.01 (29th January 2024)

• JS Fix for new imports to load default values for postmeta

1.00 (29th January 2024)

• Very first working version including interface
  • please read the knowledge-base guide to understand how to use this, in particular how to use the function system