reinteractive has been working with Westfield Labs, the digital arm of the world’s largest shopping centre group, since 2012. As part of this work, the reinteractive team has been responsible for the integration of many partner retailers with Westfield’s “online mall”, including retail giant David Jones.

reinteractive were approached by the Westfield Retailer Integration Team to create a system that could consume retailer data from a wide variety of sources, parsing and mapping the data to conform to Westfield’s conventions (such as colours and categories) so that it could be displayed on Westfield’s “online shopping mall”. The end result is a single website that allows users to browse thousands of products from dozens of individual retailers’ online stores.

One of the major challenges of this project is that each retailer had their own system for providing their product data, while some didn’t provide data at all. Our team built processes around each data source to gather data on varying schedules, or with no schedule at all. We received, gathered and parsed the data files and their formats in many different ways. Data sources include connecting to and downloading CSV files from SFTP servers, receiving posted data from an API endpoints including the Google Product Feed and even using HTML scraping of individual retailer websites with Anemone and Nokogiri to gather data.

Our team also needed to improve the efficiency of the system (both memory and speed) in order to quickly process larger product data sources, such as the massive product catalogue of David Jones.

The reinteractive team worked with Westfield Labs’ leading edge internal systems and processes, using GitHub to host and review changes to the Westfield code and Jenkins to run continuous integration and deployment, with automatic execution of test suites utilising RSpec and Capybara, before an RPM package was built and deployed by the Westfield Labs Operations Team. Westfield's Service-Oriented Architecture approach of using REST APIs for all of their internal and client facing apps, meant we could minimise the initial learning curve required to get up and running and iterate quickly by having autonomy over our own services within established SLA's.

As a remote contractor to Westfield Labs, the reinteractive team recognised the importance of keeping in high communication and made extensive use of online communication tools throughout the project. Each morning the reinteractive team met with Westfield’s Retailer Integration Team via Skype to discuss the work that was done the day prior, and to plan the day ahead. Throughout the day, any additional questions were communicated over email, HipChat and Skype and JIRA, adhering to communication practices requested by Westfield Labs.

Westfield is positive proof that Agile practices are appropriate and able to be implemented effectively and efficiently in an enterprise environment.

Ongoing Work

Westfield Labs continue to work with our team and 2014 has seen our responsibilities for their code base grow. As well as Westfield’s retailer integrations, the reinteractive team have done ongoing work to provide a new metrics service to assist the Westfield Labs team in tweaking the algorithms for their online malls.

By embracing the agile methodology and leveraging the reinteractive development team through remote work, Westfield Labs have been able to deliver new software and systems rapidly to meet their ever-changing market and evolving technology requirements to stay ahead in their industry.