Integrating Woocommerce with diamond data from Rapnet or IDEX

Update Dec 2016
Another approach in integrating large data in Woocommerce store was tried successfully through Restful web services. A separate web service application on CodeIgnitor is built and hosted on the same server.
The application supplies data to Woocommerce based e-store as and when required. In Our case, we had to push about half a million diamond stone data into a Woocommerce based e-store. 
 The diamond data  gets updated every 4 hours; thus adding so many products 6 times a day on Woocommerce was almost impossible through native Woocommerce CSV upload.
We took the route of service oriented architecture; wherein product data was supplied to Woocommerce as a service; as and when required through a robust CodeIgniter based web service.
The app easily updated half a million diamonds every four hours.In addition, through web service many more operations such as pagination, filtering and sorting were handled.
Only when user clicks an event such as "add to cart" or "add to wish list" product gets saved in Woocommerce native tables 
 

Many Diamond &  jewellery websites developed on Woocommerce ( A WordPress e-commerce plugin) require integrating diamond data from diamond trading exchanges like Rapnet, IDEX, MID or Polygon.

Unlike Magento Opencart or Prestashop which are built ground up for e-commerce, Woocommerce is a plugin on WP, to improvise it to an ecommerce store.

Essentially the “posts” in WP are altered by woocommerce to show as items/products, this makes it restrictive for customization. Furthermore for diamond data (typically Rapnet feed may go into 50,000+ diamonds) with numerous diamond specific attributes like carat weight, color, fluorescence, clarity, cut etc; integration becomes complex.

As a specialist in Diamond Jewelry websites ; Transpacific regularly receives project requests to integrate Woocommerce or WP-ecommerce with Rapnet data as well  as to build front end diamond search interface for build a ring feature.

The biggest problem:  Inserting bulk (more than 100k) records in Woocommerce-WP

How is it done:

a) Prepare the database to handle additional diamond attributes

CSV format Diamond data is downloaded from trading exchanges like Rapnet/IDEX/ Polygon or MID. Unlike a other products a diamond has many specialized attributes including cut, Florescence, table , symmetry etc.  all these  are  necessary for creating front end “diamond search engine” So the first step to prepare database by creating additional tables for diamond specific attributes.

b) CSV parsing and inserting extensions

WordPress has few good quality extensions available for  CSV import . However during our test runs most of them could  not go beyond inserting 10k+ products.

All of these flashed error.

“Fatal error: Allowed memory size Exhausted”

This happens due to WP native data insert functions to insert product data including   wp_insert_post();     wp_set_object_terms(); -      add_post_meta(); -      update_post_meta();

Consume extensive server memory ; by the time they reach 8K-10K product inserts “memory exhausted error shows up”.  

Out-of-Box approach for 100K CSV data insert.

An out of box approach had to be taken for inserting 100K products by developing a separate custom plugin  which parsed the csv files through Core PHP queries .TransPacific’s specialized module efficiently inserts about 100K product records in about 17-20 minutes.

Front end diamond search engine:

Front end Diamond search engines forms an integral part of the Diamond web-site. A spate WP plugin had to be built for diamond search which queries more than 100,000 diamond records on more than 10 filtering criterion.

Live Demo of woocommerce integrated with diamond data.

A demo is available on http://woocommerce.transpacific-software.com

 

Do contact us on connect@transpacific.in for more details.