Back to Liebermans.net

Lieberman's DataFeed

The new Lieberman's data feed is now available and contains several changes from the old data in both format and delivery mechanism. These changes are described below.

DataFeedClient Sample Application

We have provided a sample application, including source code, which demonstrates how to use this API. The DataFeedClient sample application is a windows console application which takes command parameters telling it to either download data records or images. Although this is an open source sample, this application can be used in its current form to download your data updates without any modification. You can choose to use the data feed client application out of the box or use its source code as an example to craft your own custom data feed client application.

The DataFeedClient sample application can be downloaded here:
Download Executable Only (v1.2.0.4)
Download Source Code (c#) (v1.2.0.4)

The Data Feed Client application runs on the .NET 4.6.1 Framework, and requires the Microsoft WSE 3.0 library (this library is included in the download). You can download these at:
.NET Framework 4.6.1: https://www.microsoft.com/en-us/download/details.aspx?id=49981
Microsoft WSE 3.0: http://www.microsoft.com/downloads/details.aspx?FamilyID=018a09fd-3a74-43c5-8ec1-8d789091255d&displaylang=en

Note: If you choose to download and install WSE 3.0, please choose the "Runtime" installation.

Using the Data Feed Client

Retrieve the data feed baseline records:
dfclient.exe -get:baseline -recordspath:c:\targetdir
 
Retrieve the data feed images (all publishers, requires about 30GB available space):
dfclient.exe -get:images -imagespath:c:\targetdir\images
 
Retrieve the data feed images (single publisher only):
dfclient.exe -get:images -imagespath:c:\targetdir\images -publisher:code
 
Retrieve data updates
dfclient.exe -get:records -recordspath:c:\targetdir -recordschangedsince:2010-03-10
 
Get current inventory levels:
dfclient.exe -get:inventory -recordspath:c:\targetdir
 
Get detailed publisher data:
dfclient.exe -get:publishers -recordspath:c:\targetdir
 

You must have a valid web service credential in order for the DFClient application to call the web services and access the data. If you do not have a web services credential, please contact Lieberman's at developersupport@liebermans.net to apply for one. Once you have your web service credential, fill in the values in the app.config file included with the DFClient application as follows:

<appSettings>
<add key="WebServiceCredential.ApplicationToken" value="<insert here>" />
<add key="WebServiceCredential.EmailAddress" value="<insert here>" />
<add key="WebServiceCredential.Password" value="<insert here>" />
</appSettings>

The Underlying Data Feed API

Data Records

The data records baseline can be downloaded through the web service API by calling the GetDataFeedBaseline method. The reason we have made the baseline available through a web service call rather than putting it up for download at an FTP location is for security purposes. In order to call the web method to download the baseline you must have proper access and each call is logged, which helps us ensure the data stays within registered Lieberman's customers. The provided sample application will download the baseline for you so no programming is necessary in order to retrieve the baseline. Note that the baseline includes the data records only, images must be retrieved separately as described below.

Updates to the data can be accessed on demand by a web services API located a https://services.liebermans.net/api/v2/LiebermansService.asmx. The API works by passing the GetDataFeedRecords web method a date and it returns all records changed on or after that date. This method sends the changed records as a ZIP file in byte array via the web service output. The ZIP file contains the data records in the same format as the baseline, just a smaller set of data which was changed in the date passed.

It is important to note that the categories and search keywords downloaded as updates by th GetDataFeedRecords method are the complete list of categories and keywords for the artwork records included in that update. This means that if a category association is removed or if a search keyword is removed, we do not send a special message to indicate to remove that category or keyword. Instead, they are simply not part of the categories or keywords passed back in the update. This means that when using these records to update your database, you should first update the artwork records provided in the update, then remove from your database any categorization or keyword data for the artwork records in the update, then insert the categories and keywords provided in the update. If you do not process the data in this way, you will not pickup any removed categories or keywords until you receive the next baseline. To give a clear example of this, let's say a piece of art was incorrectly categorized in the "Dogs" category, but in fact it's a cat. Since your last update, our staff has corrected the problem and placed this piece of art into the "Cat" category and removed it from the "Dog" category. When you request the data update, the data will now indicate that the art is in the "Cat" category and have no reference to the "Dog" category at all, however it will not explicitly tell you that it was removed from the "Dog" category. This is why you need to remove your category and keyword associations for each record included in the update prior to inserting the new data into your database. Additionally, each time you process an update you should always scan the category definitions (categorydef.txt) for any new categories and insert them into your database.

This method and others in the API which transmit large amounts of binary data use the MTOM specification, supported by the W3C and included in Microsoft's WSE 3.0, which uses XOP (XML-binary Optimized Packaging) to send data in base64Binary format without the use of MIME or DIME. We chose this format as it has better cross platform support than Microsoft's DIME.

Because of this on demand method of retrieving data updates, it is recommended that you schedule a task on your server to download updates each day. Because of the nature of this architecture, you can even schedule intra-day updates to have the most up-to-date data. The GetDataFeedRecords method will only accept a date of up to 3 months in the past, and no earlier than the date of October 2008 where the first data baseline in the new format was published.

Images

Images are provided in same manner whether you are trying to download a baseline or updates. The web services API we provide contains methods to synchronize images between our master copies on our server and your local copies on your server. The GetDataFeedImageManifest method works by providing unique MD5 hash codes of the master images on our servers so they can be compared to the image files on your server. If the file is either missing or different on your server, this is an image which needs to be downloaded. Images can be downloaded by the GetDataFeedImages web method, which uses MTOM to send up to 25 images per call as web service attachments.

The sample data feed client application calls this web method to download the images and generates thumbnail images on the fly on your local machine.

It is important that you always request an image update whenever you request a data update, otherwise you may have a data record without an image.

Data Feed Baseline Files

Click here to download sample data.

artwork.txt

Column Description
PID Product ID. The unique identification number assigned to each piece of art.
PUBLISHER Three character string publisher code.
PUBITEMNUM Publisher Item Number. The unique identification number assigned by the publisher. This number is normalized (see below) by removing all non-alphanumeric characters.
TYPE Type of the artwork.
ISAVAILABLE A boolean value indicating if the artwork record is available to purchase (1 = TRUE, 0 = FALSE).
CANCANVAS A boolean value indicating if the artwork record is eligible to be transferred to canvas (1 = TRUE, 0 = FALSE).
CANFRAME A boolean value indicating if the artwork record is eligible to be framed (1 = TRUE, 0 = FALSE).
ARTISTLN Artist last name.
ARTISTFN Artist first name.
TITLE Title of the piece of art.
PPRWIDTH Paper Width.
PPRHEIGHT Paper Height.
IMGWIDTH Image Width.
IMGHEIGHT Image Height.
RETAILPRICE The retail price of the artwork.

categories.txt

Column Description
PID Product ID. The unique identification number assigned to each piece of art.
CATEGORYTYPE Type of the category.
CATEGORYID The unique identification number assigned to each category.

category_def.txt

Column Description
CATEGORYID The unique identification number assigned to each category.
CATEGORYTYPE Type of the category.
PARENTCATEGORYID Parent category ID number for building a category tree.
CATEGORYNAME Name of a category record.

keywords.txt

Column Description
PID Product ID. The unique identification number assigned to each piece of art.
SEARCHKEYWORD A keyword associated with an artwork record.

Changes In Data Format

The new data format has changed from the old format in the following ways.

Tab Delineated - The new data files are tab delineated instead of comma separated.

New Column Headers - Some of the column headers have changed.

Normalized Publisher Item Numbers - The publisher item numbers are all normalized. Any spaces or special characters such as dashes or periods have been removed, as well as any leading zero's. For example, a publisher item number "1-100" normalized is "1100" and "0100" is "100".

Unavailable Items - The data feed contains a column, IsAvailable. This column indicates if a print is available for sale or not. If an item is not available, it has either gone out of print and become discontinued or the item has been removed from the Lieberman's catalog and is no longer available through Lieberman's. All records in our database are included in the data feed, however many of the older items are marked as not available. Records are never deleted, rather they are marked as unavailable. If an item becomes discontinued, it will be included as a changed item in the data feed and marked as not available. This is how you process discontinued items, a separate discontinued list is not provided.

PID - A new identifier is introduced in this data feed, the PID (Product ID). This is a single identifier which can be used as a primary key to uniquely identify a single product or piece of artwork. Although we continue to provide the publisher code and publisher item number, our internal system is based on PID as the method of accessing artwork. This is more beneficial than the publisher code and item number combination as publisher item numbers are subject to change (although this rarely happens). The categories and keywords data files use PID as the artwork identifier. As you transition your systems to use PID as the primary identifier, you may want to build a function which looks up the PID given the publisher code and item number.

Type - The data feed identifies the type of the artwork. Possible values are PRINT, POSTER, GICLEE, HANDCOLORED, SERIGRAPH. Art prints (PRINT) are generally printed on high quality paper while wall posters (POSTER) are printed on lower quality paper, which are typically your movie posters, etc.

Categorization - A little history may be necessary to explain the change in categorization. Lieberman's in the past had a small list of categories provided in its data feed. Metaverse Corporation had its own list of categories. When Metaverse acquired Lieberman's, we had to make a choice on which categorization tree to continue to support because of the high cost of manual labor involved in categorizing the images. The Metaverse category tree is more detailed and we believe more valuable for merchandising the images on your websites. Category types provided are by subject, genre, and color. The style, decor, and secondary color category types are no longer supported, however the list of search keywords will still be supported. In the past, the search keywords contained the publisher item number as one of the search keywords. Going forward, this will not be included. If you would like to program your search bar to work on publisher item number, that data is provided on the artwork record and we decided not to duplicate it in the search terms.

In the new categorization system, artwork can belong in multiple categories and not limited to a single category. Instead of a flat list, this system is hierarchical - categories have a parent / child relationship. The category_def.txt file in the data feed tells you what the parent categories are for each category, or -1 for no parent category, meaning a top-level category. When displaying categories on your website, you will need to take the parent / child relationship into account. For example, the category for birds is simply called "Birds" with parent category "Animals", not called "Animals:Birds" as the old format did.

We understand that much of your SEO efforts are based on the existing category ID's and URL's and it is important to both your business and ours that you maintain your existing SEO placement. To aid in this we have provided a mapping of the old Lieberman's categories to the new Metaverse categories. We suggest that your website code incorporate this mapping so that the URL's for your existing categories do not have to change. It is important that you take the time to ensure all of your category URL's continue to point to the same categories. The old categorization data which you already have is still valid however updates in this format will no longer be provided. We apologize for any additional work this may cause you however going forward you have access to a more flexible, more detailed category tree, which should help grow your business.

The map of old category IDs to new IDs can be downloaded here:
LiebermansOldCategoryMap.xls

It is also important to note that the new category list is not static. New categories are routinely introduced. The category definition table is provided in the baseline and each time you request a data update. Be sure to process this file and create any new categories in your database.

Usually Ships Time - The GetPublishers web method of the API provides a list of publishers and attributes. One attribute is UsuallyShipsInDays. This varies from publisher to publisher depending on how often we are placing orders with these publishers. Publishers which are ordered every day are marked as "Usually Ships Within 2 Days" as we receive the product on the next business day after we order it. Some publishers are shipped to us via 2-day express from the west coast and some internationally. The usually ships time indicates approximately when we expect to ship the item from our facility from when the order is placed with us. You may want to use this data on your website to give customers the best indication on when they can expect to receive their order. Note that this data changes from time to time. For example, around the holidays we order more frequently and overnight shipments to us. We will change the usually ships time to more favorable terms to reflect this, which improves sales. In order to get these updates, you will need to periodically call this website and update your database with the latest ship time.

Inventory Levels - The API provides a method GetInventoryLevels, which returns the inventory level for all items which are currently stocked (items which we do not currently have in stock are not included in this list). We indicate the stock level as High or Low. If the stock level is "High" you are fairly assured that we will be able to ship your order for this item from our local stock. If the stock level is "Low" it means we have quantity on hand but if another order comes in before yours, that stock might be assigned to another order. Items that we have in stock usually ship the same day they are ordered if we receive the order by 4pm EST that day. We recommend that you call GetInventoryLevels and update your database with these once a day. This gives you the ability to state on your website that these items will ship the same day, a major competitive advantage. You can also call the GetInventoryLevel method, which will return the inventory level for just one item. This can be used if the stock level is Low and you want check again when a customer puts this item into his shopping cart, to make sure the item still has stock.

Some publishers we carry are 100% stocked, however for most we stock a subset of the best sellers. To display the best "usually ships" statement on your website, first check inventory levels to see if you can state it will ship the same day. If it isn't in stock, then fall back to the UsuallyShipsInDays value given from the GetPublishers method.

You may also notice that the Publisher object returned by the GetPublishers method contains a Boolean value StockReplenishmentOnly. These are publishers which we stock everything from. Unlike regular publishers which we order daily from, stock replenishment only publishers are ordered about once a month in much larger quantities. In between our large stock purchase orders, it is possible for some of the items for these publishers to sell out of our local stock (they are still available from the publisher, only our local on hand supply is temporarily out). If you place an order for such an item, it will cause us to place a stock purchase order much sooner so that your customer is not waiting on the item. This may cause some delay but the order will be processed. It is not essential for you to take any action on this information. However, to provide the best experience to your customers, we recommend that you temporarily remove any item from your website from a stock replenishment only publisher which we do not currently have on hand stock of. This can be accomplished by querying the inventory levels and examining the contents for the stock replenishment only publishers.

 
 

Lieberman's

Home
About Us
Contact Us
Become a Vendor
Lieberman's Blog
Developer API
Customer Feedback

Our Products & Services

Prints & Posters
Canvas Transfers
Posterhangers
Database / Drop Shipping
Bargain Packs

Browse Artwork

Advanced Search
Art By Collection
Art By Subject
Best Sellers

Lieberman's
49 Stouts Lane
South Brunswick, NJ 08852
A Metaverse company
© Liebermans.net. All rights reserved.