X.Commerce Introduction Part 3: Integrate technologies with X.Commerce

Fri, 12/16/2011 - 08:46

Following the X.Commerce introduction series, in this post I'm going to talk about integration with X.Commerce, this is a very important aspect of X.Commerce because integration is always hard... 

Since I started working as a developer, almost every time, integration with another application was a bit of a pain, there is always something to worry about and of course... something that doesn't work as expected within the communication channel.

Integration is a very important aspect of X.Commerce. The whole idea is to connect and relate different applications through the fabric and standardize the way that applications talk to each other. In this post I'm going to explain how easy it is to integrate three different technologies (PHP, Java and .Net) with X.Commerce and end up victorious!

By the way, I am a rookie with Java, and to be honest, not even close to a PHP developer; so in the very beginning I thought this was going to be no less than nice a sequel of Mission Impossible. The truth is, it wasn't at all! It is very easy to do. There are some considerations to have in mind, so that's it... and I always wanted to say this... Start your engines:

Before we start, make sure you understand what X.Commerce is, you have the Fabric up and running and have a capability registered, if you don't know any of these, then please, read the previous articles (the first one here, and the second one here).

In these examples we are going to mix three different technologies: 

  • A Magento Storefront as the starting point of our event (so I asume you already have a Magento running)
  • A .Net Capabilty that receives messages for the "/Inventory/update" topic  
  • A Java Capability that receives messages for the "/Inventory/update/success" topic.

For this example we are going to use the Innovate developer demo from the Official X.Commerce GitHub, if you don't have it you can get it from here. To make this demo work, you will need to get Avro.php, you can get it from this package, navigate to the path "x-commerce/assets-php/php/avro" and you will see an Avro.php and an Avro folder.

1. Fabric Configuration
Our first step is to configure our Fabric, for this demo we need to have two topics, two capabilities and one tenant, in my previous post I wrote how to do this so you can do it very easily, this is what we need:


  • /inventory/update
  • /inventory/update/success


  • TestMerchant


  • TwitterPublisher (.Net) -> Subscribe it to the /inventory/update topic and authorize TestMerchant
  • TwitterListener (Java)-> Subscribe it to the /inventory/update/success topic and authorize TestMerchant

When creating the Capabilities leave the EndPoints in blank since we are going to update them later.

2. Install a Magento Extension

Inside the Innovate developer demo you will find an extension folder, to make this demo faster we will reuse this code, so lets install this extension.

  • From the Package copy "extension\app\etc\modules\Xcommerce_InventoryPublisher.xml" to "app\etc\modules" in your magento.
  • From the Package copy the entire folder "extension\app\code\local\Xcommerce" and its sub folders to "app\code\local" in your magento.
  • Get Avro.php and Avro folder from "x-commerce/assets-php/php/avro" inside the "xdevcamp-sf-2011" and copy to "lib" folder in your magento. I want to thank my buddy Roque for helping me with this.. since the innovate developer demo does not specify to add this library, and without this Avro in your lib, the extension won't work, as I told you before... I'm not a PHP developer.
  • Follow the next steps to add a new attribute:
    • Go to the Magento dashboard (e.g. http://localhost/index.php/admin/dashboard)
    • Go to Catalog/Attributes/Manage Attributes
    • Click "Add New Attribute"; under Properties, enter (leave others as defaults)
      • Attribute Code: deal_of_the_day
      • Catalog Input Type for Store Owner: Yes/No
    • Under Manage Label/Options, enter
      •  Admin: Deal of the Day
      •  Default Store View: Deal of the Day
    • Click "Save Attribute"
    • Go to Catalog/Attributes/Manage Attribute Set
    • Edit attribute set "Default"
    • Drag deal_of_the_day into the General group
    • Click "Save Attribute Set"

This is everything we need to install the extension, by the way... I want to thank Lucas for helping me with the magento installation.

3. Modify the Magento Extension

This extension uses the save catalog event, it takes the modified item and sends a message for the topic "/inventory/update" to the Fabric, this extension is prepared to send avro binary messages to the fabric and receive it with a Ruby capability, so we are going to slightly change it.

First of all, the Eclipse plug in and helpers form XCommerce are not yet 100% done, se there are some problems when sending avro binary messages, it can't parse it very well, so after struggling with this for a few days, I realized that AVRO is just a json representation of an object with its schema attached. So we can send the json object without the schema inside and it will work too, the good thing about that is that you free your application from knowing about avro at all... the bad thing is that your capability will need to know what object it will receive in the incoming messages, which is not really a bad thing for me, I mean you know what object you will receive according to the topic of the message... so we will change the observer to just send a json object instead of sending an avro message.

Here Is my observer's code:

{syntaxhighlighter brush:php} include_once 'avro.php'; class Xcommerce_InventoryPublisher_Model_Observer { public function saveInventoryData($observer) { $p = $observer->getEvent()->getProduct(); $attributes = $p->toArray(); $item = array('sku' => $p->getSku(), 'title' => $p->getName(), 'currentPrice' => $p->getPrice(), 'url' => $p->getProductUrl(), 'dealOfTheDay' => $attributes["deal_of_the_day"]); $items = array("items" => array($item)); $dw->append($items); $dw->close(); $tenantAuth = "QVVUSElELTEAzC4ztOZHXF+wsnCJhoQiQX7tlYyD9xpUADHXL4ZZ0P4="; $json = json_encode($items); Mage::log($json, null, "log", true); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://localhost:8080/inventory/update'); curl_setopt($ch, CURLOPT_HEADER, true); curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); curl_setopt($ch, CURLOPT_TIMEOUT, 10); curl_setopt($ch, CURLOPT_POST, TRUE); curl_setopt($ch, CURLOPT_POSTFIELDS, $json); curl_setopt($ch, CURLOPT_HTTPHEADER, array("Authorization: Bearer $tenantAuth", "Content-Type: application/json")); $response = curl_exec($ch); $error = curl_error($ch); $result = array( 'header' => '', 'body' => '', 'curl_error' => '', 'http_code' => '', 'last_url' => ''); if ( $error != "" ) { $result['curl_error'] = $error; } curl_close($ch); } } {/syntaxhighlighter}

Make sure you have the correct Fabric Url and Bearer Token, with this extension, after saving a product, it will send a json pretty close to this:


4.Create .Net Capability

Creating a Capability is not hard if you know how the Fabric works, but when working with .Net there are no helpers at all to do it so I needed to figure out the best way to get the job done, I don't know if it is the best way but at least it is what worked for me Smile, this will be the "TwitterPublisher" Capability.

Basically, the Fabric sends an incoming message to all subscribed capabilities with this pattern {Capability Endpoint}/{Message Topic}, for example, http://localhost/Inventory/update, this is exactly an MVC type of url, so I chose the MVC framework to create this capability.

It is pretty easy:

  • Open Visual Studio and create a new MVC Project
  • Configure your Project to run on an specific port, to do this, right click your project, on the Web Tab, select Specific port and type whatever port you are comfortable with.
  • Right Click the Controllers folder and add a new Empty Controller "Inventory"
  • Add a new Action Update
  • Open your Fabric Manager, select the Capability "TwitterPublisher" and set the new Endpoint i.e.: http://localhost:2150

That's it, now every time you send a message for the Inventory/publish topic, it will be sent to this url.

The next step is to parse the message and be able to respond back to the Fabric, in order to do this, we need to have the classes for the expected objects the capability will receive. This can be autogenerated (also in Java), you need to download the Avro package from apache, you can get from the X.Commerce mirror, navigate to lang/csharp/src/apache/codegen and open the c# project, rebuild it to generate the code generation executable.

In a console, navigate to csharp/build/codegen and run the following command avrogen.exe -p {sourceAVDL} {destination}, this command will generate all necessary entities for our message based on an AVRO IDL file... this file is inside the Innovative developer Demo, and it is called Inventory.avdl, so replace the {sourceAVDL} with the full path to this file, the destination parameter would be your project path, after that, refresh your project and include the new folder "com".

Now we have everything necessary to parse the incoming message, first of all, you need to check the authorization header, this is can be done with just a Request.Headers.Get("Authorization"), in this demo I'm just getting the value but doing nothin with it, how authorization logic works depends on every project so this is the way to get the incoming bearer Token, how you use it is up to you Laughing.

As I told you before, we are sending our message from Magento as a simple Json string, so getting these values is pretty easy, I'm using the Newtonsoft library to deserialize the Json request into the object I'm expecting, this capability should do something with Twitter, like posting a new Tweet every time we get an updated product, to keep the code clean I'm going to leave that aside so we don't mix different concepts, do you can do whatever you may like :)

Now we need to send a response, the idea is that our Magent send us an Items object with items Inside... do whatever we want with them, and send a response with an UpdatedItems objects with the very same items that we received, as UpdateItems and Items have the same structure, we can send back to the fabric the very same Json we received.

{syntaxhighlighter brush:csharp} public class InventoryController : Controller { // // GET: /Inventory/ public ActionResult Update() { string auth = Request.Headers.Get("Authorization"); System.IO.Stream str; Int32 strLen; str = Request.InputStream; strLen = Convert.ToInt32(str.Length); byte[] strArr = new byte[strLen]; Request.InputStream.Read(strArr, 0, strLen); string json = new System.Text.ASCIIEncoding().GetString(strArr); Items items = JsonConvert.DeserializeObject(json); //Do whatever twitter thing you want =) UpdatedItems updated = new UpdatedItems(); updated.items = items.items; //byte[] strResponse = new byte[strLen]; //Avro.IO.BinaryEncoder encoder = new BinaryEncoder(new MemoryStream(strResponse)); //Avro.Specific.SpecificWriter writer = new Avro.Specific.SpecificWriter(updated.Schema); //writer.Write(updated, encoder); HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost:8080/inventory/update/success"); request.ContentType = "application/json"; request.Headers.Add("Authorization", "Bearer QUkAAW9gX/24Ff7YcB3ZDbE0FptcNxAcGCBf1pc/Sqx128fqFWcXJxtNNI04AgD0+vwp7w=="); request.Method = "POST"; Stream requestStream = request.GetRequestStream(); byte[] responseBytes = new System.Text.ASCIIEncoding().GetBytes(json); // now send it requestStream.Write(responseBytes, 0, responseBytes.Length); requestStream.Close(); HttpWebResponse response = (HttpWebResponse)request.GetResponse(); //Do whatever you need with the response return new HttpStatusCodeResult(200); } } {/syntaxhighlighter}

That's everything we need for our .Net Capability.

4.Create Java Capability 

Creating a Java Capabilty is a trivial task, with the Eclipse Plugin you can do it very quickly and easily, just click the X.commerce button and select the Option "New Capability"

Fill the fields Project Name with "TwitterListener" and Package with whatever you like, then select the Inventory.avdl file from the innovate developer Demo package, then click Finish. 

That's it, it is done... easy right? Laughing

The next step is to fill the capability info form the Fabric Manager, just navigate to WebContent->WEB-INF->classes-> and fill in the fabric, tenant and capability bearer tokens.

To test this Capability, right click the Project and Select Run as -> Local Capability, this will start the fabric automatically and update the endpoint for this capability.

As we are using the same avdl file for both capabilities, the Java Generator, will create handlers for both "/inventory/update" and "/inventory/update/success" events, so you can get rid of the first one... if you don't, then both .Net and Java capabilities will receive the update message, which is not wrong... it is just an implementation decission Wink.

One last thing, to test that the capability gets the message correctly, replace your onInventoryUpdateSuccess method for this one:

{syntaxhighlighter brush:java} protected void onInventoryUpdateSuccess(XFabricMessage message) { try { UpdatedItems updated = (UpdatedItems)message.getMessage(UpdatedItems.SCHEMA$); System.out.println("Updated Items: " + updated.toString()); } catch (IOException e) { e.printStackTrace(); } } {/syntaxhighlighter}

5.Testing the whole Process

The last thing we need to do is to try this out, which is the easiest part.

Go to your Magento admin, open the catalog and edit any item, this will fire the observer event and the PHP code will send the first message to the Fabric, the Fabric will redirect the message to the .Net Capability, that will send an update success message to the Fabric, that will redirect it to the Java Capability, and that's all! Victory!

I hope this helps anyone who is trying to test this kind of integrations, I've working (or fighting...) on this for about 2 weeks and as there isn't much information yet, mostly on .Net, It was not a really easy task, but once learned it is very easy... I know some things can be improved but I wanted to keep it simple for your understanding.

Leave your comments and thoughts!



X.Commerce Introduction Part 1 - The Fabric

X.Commerce Introduction Part 2: Registering a Capability

X.Commerce Introduction Part 3: Integrate technologies with X.Commerce

.Net Developer
Senior developer interested in multitier architecture, web based, back-end and core app design and development.