Changelog

1.72 (17th April 2025)

• Fixed bug with product_type warning which was affecting the load-csv stage.

1.71 (12th April 2025)

• Fixed downloadable product support with new fields for download type, expiry, and limits - just add meta: _downloadable=yes and _downloadable_url to use
• Fixed attribute serialization to preserve original attribute order for update items tests
• Fixed product_type taxonomy - refactored so it uses other taxonomy stages
• Refactored taxonomy and postmeta processing loop to use new function to guarantee retrieval of all but exactly the columns required from the imports tables
• Fixed meta updates on 2nd runs and added tests for posts, taxonomy and postmeta updates to different values

1.70 (11th April 2025)

• Improved auto collation handling - the value back from $wpdb->get_charset_collate is often not what is in use in the database, and for importing, being able to join the tables without adding a collation override is important
• Cleaned up a lot of code, mostly involving switching to using the class properties instead of duplicating them into local variables
• Fixed duplicate image upload issue if duplicates were inside the galleries

1.69 (10th April 2025)

• Fixed bug with WooTemplate extending PostTemplate where Woo child was replacing rather than adding to postmeta to update
• Added automated test to confirm custom meta like above is loaded and available

1.68 (8th April 2025)

• Updated JS to show special fields under Post Fields section - these will probably move somewhere else soon, but for now they are back

1.67 (8th April 2025)

• Enhanced image handling:
  • Fixed featured image import to correctly grab the first image if pipe separated
  • Fixed broken code in image gallery upload, previously images were only importing correctly for galleries if you used delayed image import
  • Increased timeout for image downloads from 100ms to 3000ms - if this slows down your import, use Delayed Image Import
• Fixed config export to prevent JSON double-encoding issues
• Added enhanced export functionality for testing:
  • Added featured image URLs to exports
  • Added gallery image URLs to exports
• Added new simple products test with images & gallery images

1.66 (4th April 2025)

• Added .gitattributes file to exclude tests folder from released zip file

1.65 (2nd April 2025)

• Enhanced user interface for better usability:
  • Redesigned file upload interface with improved visual feedback
  • Auto open sections at relevant times, e.g. upload CSV opens the CREATE IMPORT section
  • Added toggle to show/hide unused fields for cleaner interface, defaults to hide when reloading previously saved imports
  • Improved taxonomies sorting in template selection (prioritizing Post and WooCommerce templates for new imports)
  • Other CSS improvements in admin interface
• Added support for CSV files with full paths from CLI imports tool into the admin interface
• Enhanced testing infrastructure:
  • Added verbose mode to test scripts with optional detailed output (--verbose flag)
• Added human-friendly descriptions for postmeta fields

1.64 (1st April 2025)

• Created comprehensive tools management system with standalone classes
• Added new CLI commands for better import/export workflow:
  • Export: wp ssi export ./exports/products.csv product - Export products to CSV file
  • Export with options: wp ssi export ./exports/posts.csv --exclude-dates - Export posts excluding date columns
  • Export taxonomies: wp ssi export-taxonomies product ./taxonomy-export.json - Export product taxonomies
  • Import taxonomies: wp ssi import-taxonomies ./taxonomy-export.json - Import taxonomies from file
  • Load config: wp ssi load-config 10 ./import-config.json - Load config into import ID 10
  • Save config: wp ssi save-config 10 ./export-config.json - Save config from import ID 10
  • Create import: wp ssi create "New Import" ./products.csv WooProductTemplate - Create new import
• Enhanced export functionality with options to exclude date columns
• Improved SQL generation for more consistent and reliable exports
• Added SQL syntax error detection to prevent failed imports
• Improved automated testing infrastructure with better error reporting
• Added comprehensive test suite for regression prevention

1.63 (31st March 2025)

• Added new CLI export command to help with automated testing: wp ssi export $post_type $file_name - no config, it just exports everything from wp_posts, wp_postmeta and all the terms for that post type
• Fixed big issue with fix-attributes - this was only working on 2nd runs, not sure how long for, tests added to prevent it ever happening again
• Fixed issue with hierarchical taxonomies - child leaf was not getting the correct term_taxonomy_id until UPDATE (2nd run)
• Added comprehensive tests to wipe, run, export, re-run, export, diff - more to come to guarantee no regressions and get us to Beta

1.62 (27th March 2025)

• Fixed error handling so that if an actual SQL error occurs then it aborts properly
• Upgraded hard-error reporting so that it uses the :error from WP CLI (it shows in red in the console rather than white)
• Upgraded insert-postmeta stage so that it will skip missing meta columns from the import table which can happen, e.g. if _sale_price was not defined
• Upgraded valid-CSV detection so it works for exports from other places - instead of checking mime type, actually reads the first header row

1.61 (25th March 2025)

• Fixed bug with deletes from refactoring job

1.60 (25th March 2025)

• Added all remaining missing columns for config from wp_posts
  • defaults will be used if they are missing (e.g. author will be the current user, date will be current date)
  • any items NOT configured will guarantee those items will NOT be updated, but may be inserted as required (e.g. author, dates, other sensible defaults)
• Fixed last refactor job - it was working for posts, but not for products
• Added new CPT template in case anyone wants to experiment with post types other than post or product
• Found and fixed 1 place preventing simultaneous import execution (so long as it's different import ids)
• Found and fixed a bunch of other bugs that arose from the refactor job
• In particular, inserts always insert defaults now for any unspecified import whereas updates never will use defaults

1.59 (10th March 2025)

• Minor refactoring
• Added Post template, tested, working
• Separated all product related stuff out into the WooCommerce-specific template
• Added import history to admin interface
• Added import details to admin interface
• Hardened checks for additional options
• Fixed issue with HTML special characters in term names (flat terms & hierarchical)
• Upgraded the sanitize term function to exactly mimic wp_insert_term - should fix any remaining broken term names
• Added new import-taxonomies stage specific to posts and specific to products
• Added fallback for terms failing to insert to grab the duplicate term_id and continue the import
• Added sanitize_terms to clean up the term mess I made
• Added 2 new tables to record ambiguous term names - e.g. if Dave's stuff is a term name, that could be inserted either html encoded or not, and may not be discovered by the join
• Added handler to update the term_taxonomy_id's from the ambiguous terms tables
• Added SSI_RUNNING_IMPORT definition in case anything wants to do or not do stuff when SSI is running
• Batched up the insert-posts command into 100,000 per batch and added performance metrics to trace slow (200 minutes) performance for 2.7 million row CSV file
• Renamed save_posts to save-posts, renamed sanitize_terms to sanitize-terms
• Added --perf option to save_posts to show what's taking time, which function, which plugin

1.58 (4th March 2025)

• Code refactoring of runtime
• Improved performance reporting summary
• Improved logging
• Fixed issue with attributes to ensure it only updates attributes mentioned in the import
• Improved resilience of save_posts for the Woo Product template, now uses wc_get_product and $product->save etc
• Added extra output to end of import to inform admin of how to run the builds for regular woocommerce attributes
• Fixed issue with attribute/taxonomy import - changed to use wc_create_attribute instead of standard wordpress taxonomy code for attributes

1.57 (21st February 2025)

• Fixed force delete not working
• Fixed 'keep sold' items option not being recognised
• Added lots more logging everywhere
• Added import summary table at end of import

1.56 (7th February 2025)

• Fixed progress indicator on save_posts

1.55 (5th February 2025)

• Fixed 'permanently delete' items vs move to trash - thanks to @AdamDar for this one!
• Fixed issue with SQL to keep 'previously sold items'

1.54 (4th February 2025)

• Improved RAM usage for fix-attributes stage
• Added new execution URL which can be useful for profiling with Code Profiler Pro

1.53 (14th January 2025)

• Further improvements to image download to ensure it works on as many hosts as possible
• Fixed fatal error in some paths for image downloads

1.52 (14th January 2025)

• Increased image download timeout
• Fixed issue with source image URLs containing query parameters

1.51 (10th January 2025)

• Fixed CSV upload ajax for brand new users so that the CSV file correctly appears in the dropdown after upload
• Fixed New Import creation for brand new imports, so that the import loads properly after creation
• Fixed deletes - it was referencing the old meta key name, updated to use ssi_updated and deletes are working like before

1.50 (24th December 2024)

• Added continue-on-error functionality for CSV imports - if a row or term fails, it will be logged and the import will continue
• Fixed function editor initialization to properly use full code editor with PHP syntax highlighting

1.49 (23rd December 2024)

• Optimised load-csv stage by eliminating need for get_existing_terms at small cost of extra RAM consumption
• Small optimisation for the insert_flat_terms code by moving index creation on that table to the end of load-csv stage
• Optimised load-csv stage using more memory efficient way of reading the CSV file

1.48 (20th December 2024)

• Added version number to output so I can be sure which version clients are using - some hosts set opcache really high so it can still be using old code on some hosts even if updated through wp-admin

1.47 (20th December 2024)

• Fixed bug with hierarchies being too long - previous limit of 200 characters was lifted (no limit now other than 200 char limit for each term name in the hierarchy)
  • However, this is in contradiction with WordPress - because they uniquify the slug, their uniquify function can create slugs > 200 characters which then fail
  • There's a new filter added to wp_unique_term_slug to reduce the size of the slug and still ensure it's unique so our import can handle any number of X > Y > Z terms so long as each component term name is < 200 characters

1.46 (18th December 2024)

• Fixed memory leak in fix-attributes stage - it was from a call to WordPress core function update_user_meta - even though I only asked to update user meta, it also caches the meta for that item

1.45 (13th December 2024)

• Added PHP directive to display errors to CLI output for cases of some hosting environments having everything off
• Added better debug to failed function eval so you now find out exactly which column has a bad function defined for it

1.44 (12th December 2024)

• Fixed issue with saving PHP functions - was working if the function was still in codemirror/expanded view, otherwise was saving nothing

1.43 (11th December 2024)

• Fixed taxonomy matching bug introduced in 1.35
• Fixed save import broken if taxonomy with function (not csv mapping) was in the import
• Found and fixed a bug affecting terms with parents being mistakenly matched with parentless terms
  • this in turn caused some terms to not be attempted to import and subsequent checks to fail because of these terms being missing
• Added CSV file name for configured template to template info at top of interface
• Added links to let admins see products affected by this import
  • also added dropdown filter to edit.php?post_type=product so users can filter on there to view products last affected by X import
• Big speed boost for upsert-relationships stage - it's as fast as it should be again, e.g. 50,000 items 0.5s now compared to 5+ minutes previously
• Strengthed wp_posts update so it runs one update per column and checks which rows to set based on source being not null and not empty
• Added performance info for load-csv stage which is often the slowest stage

1.42 (6th December 2024)

• Proper fix for tab-separated files - previously I only fixed the import interface, now fixed the actual import run too to handle tab separated files properly

1.41 (6th December 2024)

• Fix for tab-separated files

1.40 (29th November 2024)

• Fix to allow spaces in CSV file names...

1.39 (29th November 2024)

• Added alert if trying to load an import to edit when the CSV file has been deleted - it's currently needed for reference
• Added early-access auto-update prevention mechanism - users have to very clearly and deliberately enable auto-updates to mess up their live site - if you want to enable auto updates on dev or staging, add define('SSI_ALLOW_AUTO_UPDATE', true); to your wp-config.php
• Added run now button to interface for small imports

1.38 (29th November 2024)

• Improved default template to remove unused fields
• Hidden tax and postmeta entries which are meant to be hidden
• Added prevent creation of import with no name
• Added duplicate import name check
• Added new cli parameter list-files - e.g. wp ssi 20 list-files - will list files compatible with this import template
• Added new cli --file parameter - e.g. wp ssi 20 --file="alternative.csv" will use import 20 but against the alternative.csv file - this will barf if the headers are different

1.37 (27th November 2024)

• Fix for new settings to clear the local transient cache when the license key is changed
• Added a new 'recheck now' button on the license key page for when users renew or upgrade

1.36 (21st November 2024)

• Tidy up and standardisation of new license/settings page

1.35 (21st November 2024)

• Speed boost for stage 1 - load-csv stage
• Fix for stage 2 - import-taxonomies - no more reporting error when no error occurred, and fixed bug with duplicate term names in different taxonomies
• Upgraded warning messages inside functions to Fatal Errors - this helps debug failing functions, just fix all the warnings so the code can run

1.34 (20th November 2024)

• Fixed broken function calls - since I added the pretty textareas, the functions were not saving properly
• Updated the JS to use time() for admin so ugosprint definitely gets the latest JS

1.33 (19th November 2024)

• Added _default_attributes hidden field to batch import table
• Use _default_attributes to store first variation from table selected per variable product
• Updates variable product parents efficiently to set the default variation using _default_attributes
• Fixed all slugs to be sanitized - for post_name and for attribute names when they are used as keys

1.32 (16th November 2024)

• Fixed JS/CSS loading issue for certain versions of Scalability Pro (shared Settings file)

1.31 (13th November 2024)

• Fixed issue with additional options (delete items missing from previous import) from appearing multiple times as you loaded each import
• Added code to hide the 'hidden' fields in posts

1.30 (12th November 2024)

• Fixed various warnings

1.29 (12th November 2024)

• Fixed bug where everything was being marked as variable product, even simple products
• Fixed new bug with postmeta _regular_price where it was checking the function rather than the stage table for whether it should upsert or not

1.28 (12th November 2024)

• update-postmeta stage adjusted to increase performance
• Updated post-update and post-insert to not set the value if the source column is null
• Added post-slug field - by default, a sanitized post_title will be used for the post slug, but you can overrule this by setting a value for the post slug - if slug and post title are not set, the post_name will not be changed
• Added variation control into the taxonomy section
• Integrated variation control with previous hard-coded variations list

1.27 (8th November 2024)

• Reduced key size for hierarchical tables to acommodate 4-byte unicode collations
• Removed fallback to try to convert from wpallimport function - it was failing badly

1.26 (8th November 2024)

• Renamed stages to useful names
• Tidied up SQL code especially since no guarantee of execution order
• Fixed some warnings

1.25 (8th November 2024)

• Commented out the eval method for function calling in postmeta

1.24 (8th November 2024)

• Added variable product capability

  • in dev, you need to configure your variation attributes with:

    define('SSI_HARD_CODED_VARIABLE_TAXONOMY_ARRAY', array('pa_size', 'pa_colour'));

1.23 (7th November 2024)

• Added new code to let templates add and alter additional options
• Added additional options section to end of import with save/load/conditional appearance
• Added 3 new delete options to let people configure how they want their deletes handled

1.22 (4th November 2024)

• Fixed bug with flat term import where import failed if term name was empty in csv

1.21 (22nd October 2024)

• Fixed hierarchical taxonomies so they are correctly set with parents in the database - also confirmed existing terms are being retrieved correctly
• Also upgraded multi-term inserts, so if you have e.g. Cat1|Cat2|Cat3 > Cat4 then the item will be allocated to Cat 1, Cat 2 and Cat 4 correctly
• Fixed term inserts and checks for terms when you have the likes of: Salt & Pepper > Salt|Salt & Pepper > Pepper, so it correctly assigns the parent for the 2nd term Pepper to be Salt & Pepper
• Updated code to use the decimal separator correctly for prices

1.20 (3rd October 2024)

• Fixed bug in wp ssi 3 save_posts command - it was just re-running all instead of doing the save_posts

1.19 (30th September 2024)

• Fixed bug for terms with html encodable text in their name (typically &)
• Improved speed by fixing collation issue
  • rather than trusting default collations in WP, i pull the collation from wp_terms since this is what ultimately is joined to on text columns
• Hard errors now correctly stop the script and output the error in RED
• Vastly improved debug and error output to help trace problems more quickly

1.18 (25th September 2024)

• Added gallery image support
• Added checks at start to ensure you have an SKU (for now, there will be other ways of identifying the same item in future)

1.17 (24th September 2024)

• Added featured image import
• Added --verbose mode

1.16 (17th September 2024)

• Fixed the collation issue some users were having
• Added taxonomy export/import as extra option - this makes it far easier for me to replicate exactly what users see
• Fixed an issue with importing terms where the term name was used in another taxonomy
• Tested multiple imports, everything imported fully

1.15 (14th September 2024)

• Added pa_ real name indicators to interface for taxonomy
• Fixed flat taxonomy issue with missing terms after import - all terms are fully loaded now

1.14 (12th September 2024)

• Improved error logging for broken functions so you can see which one needs attention

1.13 (6th September 2024)

• Added optional collation config for those who have mismatching collations in their defaults versus actual tables
  • add the following to your wp-config: define('SSI_COLLATION', 'DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci');
  • change the collation to whatever your tables are and this will fix collation issues
  • to see your table collations run SHOW TABLE STATUS

1.12 (4th September 2024)

• Fixed stupid mistake of hardcoding my dev table name into stage 2!

1.11 (4th September 2024)

• Much more work and fixes for hiearchical taxonomies
  • tested with flat and hiearchical, tested 5 levels deep
  • only setting leaf term, not all the ancestors
• Fixed attributes
  • product attributes now work properly and are loaded in stage 10
  • this stage may be removed if I can get this back into stage 1 where the array should be defined and where this should work

1.10 (3rd September 2024)

• Speed boost by eliminating duplicates in the _hieararchies table

1.09 (3rd September 2024)

• Code inspected everything related to taxonomies
  • fixed hierarchical taxonomies not being set in destination

1.08 (29th August 2024)

• Added debug for user functions
• Added check for array type to avoid potential fatal error

1.07 (28th August 2024)

• Added export/import for templates
  • export will export to file, but also gives you the export in the textarea to copy/paste
  • import involves pasting the import config in the textarea then hitting import
• Fixed error with decimal place not being sent to the server for new imports
• Changed default mapping types to 'Direct Mapping' rather than 'function' since this should be most of the cases
• Ajaxified everything to reduce confusion for new users

1.06 (13th June 2024)

• Added txt file compatibility
• Added semi-colon separator compatibility
• Tidied up the interface a little
• Added multi-hierachical term capability (so you can have 2+ product cats per item)
• Fixed other taxonomy related bugs
• Optimised the code a bit
• Found a bunch of bugs in taxonomy, fixed more
• Added multi-product_cat compatibility
• Fixed collation and charset so it pulls from $wpdb

1.05 (31st January 2024)

• Fixed additional columns defined in template
  • these are now correctly added to the flat mappings table if the user did not add them
  • the mapData function is properly called now so any template mappings or additional data massage can occur

1.04 (30th January 2024)

• Added commas as decimals handler in the Woo import template
• Added infinite time limit in case wp cli badly configured

1.03 (30th January 2024)

• Fixed import bug to handle dashes in attribute names and postmeta keys

1.02 (30th January 2024)

• Added functions editor to interface
  • also eval these functions at import time through wp cli
  • you must trust people who can edit these functions obviously

1.01 (29th January 2024)

• JS Fix for new imports to load default values for postmeta

1.00 (29th January 2024)

• Very first working version including interface
  • please read the knowledge-base guide to understand how to use this, in particular how to use the function system