Synopsis
This page is meant to show off some of the unique and strong features of the Partshub
e-commerce webiste.
Some trivia: When I started developing the Partshub website I only had about 2 months of youtube tutorials as programming experience. I was very unprepared and unqualified for the task, which is why I take pride in what I have accomplished with this website. I have learned so much and am very grateful for the opportunity & chance to work with the people at PARTSHUB (PTY) LTD.
Some trivia: When I started developing the Partshub website I only had about 2 months of youtube tutorials as programming experience. I was very unprepared and unqualified for the task, which is why I take pride in what I have accomplished with this website. I have learned so much and am very grateful for the opportunity & chance to work with the people at PARTSHUB (PTY) LTD.
Optimisation
Scale & Complexity
Partshub is a business that operates with thousands of items nationwide. It specializes in
spare parts for coffee machines but also sells items and products for
various other industries. That is to say, the website has to be dynamic and versatile enough
to handle all of these different kinds of products. Furthermore,
most products require compatibility data.
A coffee machine can be identified in a few ways. Among those are brand, model, and manufacturer codes. It was determined very early in the development process that a custom Create-Read-Update-Delete (*CRUD) system would need to be implemented. Platforms like WooCommerce, Shopify & Wix would not be performant enough to gather the data.
While the business could use Excel to create CSV format data, and doing so would be much more convenient than other website platforms for data-entry purposes, even Excel becomes painfully slow when working with tens of thousands of rows and 30+ columns. The result of these concerns? Partshub got a CRUD system which is fully JS-based, client-side-rendered and supports two-way CSV import/export. The admin dashboard features Excel-like data entry faciliation for 4+ sheets/tables. Except, it's much more performant than Excel. When the user wants to start editting a sheet, it gets downloaded from the server. Thereafter, it's all dynamic. If the user adds, updates or deletes information, it's processed on the server side and the changed state is rendered on the frontend. No page refreshes, no hanging window- just pure performance.
A coffee machine can be identified in a few ways. Among those are brand, model, and manufacturer codes. It was determined very early in the development process that a custom Create-Read-Update-Delete (*CRUD) system would need to be implemented. Platforms like WooCommerce, Shopify & Wix would not be performant enough to gather the data.
While the business could use Excel to create CSV format data, and doing so would be much more convenient than other website platforms for data-entry purposes, even Excel becomes painfully slow when working with tens of thousands of rows and 30+ columns. The result of these concerns? Partshub got a CRUD system which is fully JS-based, client-side-rendered and supports two-way CSV import/export. The admin dashboard features Excel-like data entry faciliation for 4+ sheets/tables. Except, it's much more performant than Excel. When the user wants to start editting a sheet, it gets downloaded from the server. Thereafter, it's all dynamic. If the user adds, updates or deletes information, it's processed on the server side and the changed state is rendered on the frontend. No page refreshes, no hanging window- just pure performance.
Search
Now that we know the scale of the data, how do we make it searchable for users?
Partshub opts for two main search methods. Versatile text-based and strict selection-based. See below:
The text-based search allows the user to search based on the product's name, brand, model,
manufacturer codes, and even attributes (diameters, temperatures).
The selection-based search allows the user to select the brand, sector (espresso machines,
grinders) and then the model of their machine to see all compatibile parts.
Search algorithms and performant searching are cornerstone issues in e-commerce. Since most platforms use server side rendering, whenever the user inputs or selects something, their input is added to the page's URL and the server re-computes and renders the resulting page with updated information. This is a very costly implementation because e-commerce websites average multiple seconds to load - averaging more than 5 seconds on mobile devices. For these reasons, Partshub tries to be better. The text-based and selection-based searches are both fully dynamic.
The selection-based search boasts an average 100ms latency between the user selecting and option and the page getting updated with newly fetched information from the server. While the text-based search has a client-side 300ms buffer, the processing time on the server (the time between the server getting the request and returning a response) is consistenly less than 20ms and maxes at 40ms when the search results are in the high-hundreds or thousands. This is unimaginable without a custom solution. Read more in the next section about how this is possible.
Partshub opts for two main search methods. Versatile text-based and strict selection-based. See below:
The text-based search allows the user to search based on the product's name, brand, model,
manufacturer codes, and even attributes (diameters, temperatures).
The selection-based search allows the user to select the brand, sector (espresso machines,
grinders) and then the model of their machine to see all compatibile parts. Search algorithms and performant searching are cornerstone issues in e-commerce. Since most platforms use server side rendering, whenever the user inputs or selects something, their input is added to the page's URL and the server re-computes and renders the resulting page with updated information. This is a very costly implementation because e-commerce websites average multiple seconds to load - averaging more than 5 seconds on mobile devices. For these reasons, Partshub tries to be better. The text-based and selection-based searches are both fully dynamic.
The selection-based search boasts an average 100ms latency between the user selecting and option and the page getting updated with newly fetched information from the server. While the text-based search has a client-side 300ms buffer, the processing time on the server (the time between the server getting the request and returning a response) is consistenly less than 20ms and maxes at 40ms when the search results are in the high-hundreds or thousands. This is unimaginable without a custom solution. Read more in the next section about how this is possible.
Rust
While the backend of the Partshub website is entirely in Laravel, it also makes use of a
search microservice written in Rust. Rust is a low level language where intense
optimisation is possible. The rust application I wrote is simple: It creates a HTTP server
that servers locally (no remote access). Only applications on that same server can send and
receive
information from it. Additionally, using Nginx, I can proxy/redirect all text-based search
requests to the server directly to the Rust microservice. This means that the request never
touches
Laravel. As soon as it hits the server, it's forwarded to the Rust microservice, which then
calculates and returns a response.
The microservice loads the entire products table into its memory on startup, and has an HTTP
endpoint that the Laravel site can call - upon which it refetches the table data. This way,
all the data that is needed by the search algorithm and the user's search phrase is always
already loaded in the most memory-efficient way possible - and the "cached" data updates as
new data entry happens.
In Rust, I'm able to control the data types and their sizes- down to the level of bits and
bytes. All of this optimisation makes it almost reasonable that ~3000 products and 30
columns are combed through by a search
algorithm in ~10ms.