Lieberman's DataFeed

PLEASE NOTE: THIS API DOCUMENTATION IS FOR EXISTING API USERS ONLY. NO NEW CREDENTIALS WILL BE ISSUED.

DataFeedClient Sample Application

We have provided a sample application, including source code, which demonstrates how to use this API. The DataFeedClient sample application is a windows console application which takes command parameters telling it to either download data records or images. Although this is an open source sample, this application can be used in its current form to download your data updates without any modification. You can choose to use the data feed client application out of the box or use its source code as an example to craft your own custom data feed client application.

The Data Feed Client application runs on the .NET 4.6.1 Framework, and requires the Microsoft WSE 3.0 library (this library is included in the download). You can download these at:
.NET Framework 4.6.1: https://www.microsoft.com/en-us/download/details.aspx?id=49981
Microsoft WSE 3.0: http://www.microsoft.com/downloads/details.aspx?FamilyID=018a09fd-3a74-43c5-8ec1-8d789091255d&displaylang=en

Note: If you choose to download and install WSE 3.0, please choose the "Runtime" installation.

Using the Data Feed Client

You must have a valid web service credential in order for the DFClient application to call the web services and access the data. If you do not have a web services credential, please contact Lieberman's at developersupport@liebermans.net to apply for one. Once you have your web service credential, fill in the values in the app.config file included with the DFClient application as follows:

The Underlying Data Feed API

Data Records

The data records baseline can be downloaded through the web service API by calling the GetDataFeedBaseline method. The reason we have made the baseline available through a web service call rather than putting it up for download at an FTP location is for security purposes. In order to call the web method to download the baseline you must have proper access and each call is logged, which helps us ensure the data stays within registered Lieberman's customers. The provided sample application will download the baseline for you so no programming is necessary in order to retrieve the baseline. Note that the baseline includes the data records only, images must be retrieved separately as described below.

Updates to the data can be accessed on demand by a web services API located a https://services.liebermans.net/api/v2/LiebermansService.asmx. The API works by passing the GetDataFeedRecords web method a date and it returns all records changed on or after that date. This method sends the changed records as a ZIP file in byte array via the web service output. The ZIP file contains the data records in the same format as the baseline, just a smaller set of data which was changed in the date passed.

It is important to note that the categories and search keywords downloaded as updates by th GetDataFeedRecords method are the complete list of categories and keywords for the artwork records included in that update. This means that if a category association is removed or if a search keyword is removed, we do not send a special message to indicate to remove that category or keyword. Instead, they are simply not part of the categories or keywords passed back in the update. This means that when using these records to update your database, you should first update the artwork records provided in the update, then remove from your database any categorization or keyword data for the artwork records in the update, then insert the categories and keywords provided in the update. If you do not process the data in this way, you will not pickup any removed categories or keywords until you receive the next baseline. To give a clear example of this, let's say a piece of art was incorrectly categorized in the "Dogs" category, but in fact it's a cat. Since your last update, our staff has corrected the problem and placed this piece of art into the "Cat" category and removed it from the "Dog" category. When you request the data update, the data will now indicate that the art is in the "Cat" category and have no reference to the "Dog" category at all, however it will not explicitly tell you that it was removed from the "Dog" category. This is why you need to remove your category and keyword associations for each record included in the update prior to inserting the new data into your database. Additionally, each time you process an update you should always scan the category definitions (categorydef.txt) for any new categories and insert them into your database.

This method and others in the API which transmit large amounts of binary data use the MTOM specification, supported by the W3C and included in Microsoft's WSE 3.0, which uses XOP (XML-binary Optimized Packaging) to send data in base64Binary format without the use of MIME or DIME. We chose this format as it has better cross platform support than Microsoft's DIME.

Because of this on demand method of retrieving data updates, it is recommended that you schedule a task on your server to download updates each day. Because of the nature of this architecture, you can even schedule intra-day updates to have the most up-to-date data. The GetDataFeedRecords method will only accept a date of up to 3 months in the past, and no earlier than the date of October 2008 where the first data baseline in the new format was published.

Images

Images are provided in same manner whether you are trying to download a baseline or updates. The web services API we provide contains methods to synchronize images between our master copies on our server and your local copies on your server. The GetDataFeedImageManifest method works by providing unique MD5 hash codes of the master images on our servers so they can be compared to the image files on your server. If the file is either missing or different on your server, this is an image which needs to be downloaded. Images can be downloaded by the GetDataFeedImages web method, which uses MTOM to send up to 25 images per call as web service attachments.

The sample data feed client application calls this web method to download the images and generates thumbnail images on the fly on your local machine.

It is important that you always request an image update whenever you request a data update, otherwise you may have a data record without an image.

Data Feed Baseline Files

artwork.txt

categories.txt

category_def.txt

keywords.txt

Changes In Data Format

The new data format has changed from the old format in the following ways.

New Column Headers - Some of the column headers have changed.

Normalized Publisher Item Numbers - The publisher item numbers are all normalized. Any spaces or special characters such as dashes or periods have been removed, as well as any leading zero's. For example, a publisher item number "1-100" normalized is "1100" and "0100" is "100".

PID - A new identifier is introduced in this data feed, the PID (Product ID). This is a single identifier which can be used as a primary key to uniquely identify a single product or piece of artwork. Although we continue to provide the publisher code and publisher item number, our internal system is based on PID as the method of accessing artwork. This is more beneficial than the publisher code and item number combination as publisher item numbers are subject to change (although this rarely happens). The categories and keywords data files use PID as the artwork identifier. As you transition your systems to use PID as the primary identifier, you may want to build a function which looks up the PID given the publisher code and item number.

Categorization - A little history may be necessary to explain the change in categorization. Lieberman's in the past had a small list of categories provided in its data feed. Metaverse Corporation had its own list of categories. When Metaverse acquired Lieberman's, we had to make a choice on which categorization tree to continue to support because of the high cost of manual labor involved in categorizing the images. The Metaverse category tree is more detailed and we believe more valuable for merchandising the images on your websites. Category types provided are by subject, genre, and color. The style, decor, and secondary color category types are no longer supported, however the list of search keywords will still be supported. In the past, the search keywords contained the publisher item number as one of the search keywords. Going forward, this will not be included. If you would like to program your search bar to work on publisher item number, that data is provided on the artwork record and we decided not to duplicate it in the search terms.

We understand that much of your SEO efforts are based on the existing category ID's and URL's and it is important to both your business and ours that you maintain your existing SEO placement. To aid in this we have provided a mapping of the old Lieberman's categories to the new Metaverse categories. We suggest that your website code incorporate this mapping so that the URL's for your existing categories do not have to change. It is important that you take the time to ensure all of your category URL's continue to point to the same categories. The old categorization data which you already have is still valid however updates in this format will no longer be provided. We apologize for any additional work this may cause you however going forward you have access to a more flexible, more detailed category tree, which should help grow your business.

It is also important to note that the new category list is not static. New categories are routinely introduced. The category definition table is provided in the baseline and each time you request a data update. Be sure to process this file and create any new categories in your database.

Inventory Levels - The API provides a method GetInventoryLevels, which returns the inventory level for all items which are currently stocked (items which we do not currently have in stock are not included in this list). We indicate the stock level as High or Low. If the stock level is "High" you are fairly assured that we will be able to ship your order for this item from our local stock. If the stock level is "Low" it means we have quantity on hand but if another order comes in before yours, that stock might be assigned to another order. Items that we have in stock usually ship the same day they are ordered if we receive the order by 4pm EST that day. We recommend that you call GetInventoryLevels and update your database with these once a day. This gives you the ability to state on your website that these items will ship the same day, a major competitive advantage. You can also call the GetInventoryLevel method, which will return the inventory level for just one item. This can be used if the stock level is Low and you want check again when a customer puts this item into his shopping cart, to make sure the item still has stock.

You may also notice that the Publisher object returned by the GetPublishers method contains a Boolean value StockReplenishmentOnly. These are publishers which we stock everything from. Unlike regular publishers which we order daily from, stock replenishment only publishers are ordered about once a month in much larger quantities. In between our large stock purchase orders, it is possible for some of the items for these publishers to sell out of our local stock (they are still available from the publisher, only our local on hand supply is temporarily out). If you place an order for such an item, it will cause us to place a stock purchase order much sooner so that your customer is not waiting on the item. This may cause some delay but the order will be processed. It is not essential for you to take any action on this information. However, to provide the best experience to your customers, we recommend that you temporarily remove any item from your website from a stock replenishment only publisher which we do not currently have on hand stock of. This can be accomplished by querying the inventory levels and examining the contents for the stock replenishment only publishers.