How to read and move files in cloud-hot-folder of SAP Commerce CCv2 (Hybris) ?

Few months ago, we have migrated our SAP Commerce (formally Hybris) to Patch 29 from the patch 27. We are updating patches bimonthly. So as mentioned in the release notes, we made the necessary changes and bumped up the version numbers in 'manifest.json' file and deployed the build on CCv2 environment. Everything seems to be working fine.

Luckily we haven't pushed it to Production as our internal testing was in progress, suddenly one fine day we have started facing issues with the builds, it has started failing with errors like below,

    After checking the build logs and we were clueless for some time, as the same code was working fine on the local (just like most of the times :-) ). After applying most of our troubleshooting skills on the play, we got a clue that the cloud-hotfolder-extensions zip version that we are using on local and the one that has been used by the CCv2 build system is different. Now we were not aware about any such compatibility metrics that we need to check before downloading and using cloud-hotfolder-extensions with the particular patch (Even today the only link that we are aware about is this: https://me.sap.com/notes/2817992, let us know in case you are aware about any other publicly available documentation link). Also there is no control of this zip files version that CCv2 build is going to use is in our hand, seems like SAP decides this in the backend. And possibly that was the root cause that they were using older version of this zip earlier and then at one fine day they have decided to bump the version which eventually led to our earlier-working-fine builds to started failing. 

    So we decided to open up a SAP case with their support team. After few conversations they got the root cause as mentioned earlier and asked us to provide us more details about how we are using the Azure Storage SDK and all that. They were majorly suspecting it because the SDK version got upgraded to v12 from v8, and it was major re-write from the Azure's side. So SAP has suggested to follow this link and adjust our custom code accordingly.

    Now let us understand the use-case a bit, before talking about the whole story about what we thought of an adjustments in imported library classes to complete re-write of the logic. So we are having a cronjob which runs on regular interval and moves a file (which is being pushed from the third-party service) from specific location in Azure BLOB storage to the hot-folder location. And after the introduction of Azure SDK v12, the same code which reads and moves the file from the cloud storage location is started failing. Now there is no official documentation available on the SAP's help portal which describes this changes in details. From SAP support side they have tried for some time and then suggested that since it's your custom code you need to figure it out on your own by following the office Azure documentation!

    So let's take a look on the older code first to understand how it was working with Commerce Patch 27 (or with Azure SDK v8),

    
    And then after going through multiple iterations and trying to understand the Azure documentation, we finally landed up in a working solution like below (obviously thanks chatGPT it didn't take too long :-)),

Couple of insights here,

- It would be better if SAP Commerce Cloud will provide the control over the cloud-hotfolder-extensions’ version, just like the separate integration patch version (or may be make it part of the same zip file).

- Or else if they want to continue controlling it from behind the scenes then there should be at least a notification when they change the version of it.

- The Azure's documentation which provides the reasoning of their architectural re-designing decision is worth reading: here.

Integration of SAP Hybris with Procurement System

Introduction

This kind of integration is also known as "punchout integration". For SAP Hybris eCommerce platform, it has given a B2BPunchOutAddon out of the box to support standard punchout operations.

Before getting ahead, just a bit about the term "punchout", if you check the Wikipedia explanation from here,

PunchOut is a protocol for interactive sessions managed across the Internet, a communication from one application to another, achieved through a dialog of real-time, synchronous cXML messages, which support user interaction at a remote site. This protocol is most commonly used today in the form of Procurement PunchOut, which specifically supports interactions between a procurement application and a supplier’s eCommerce web site and possibly includes an intermediary for authentication and version matching. The buyer leaves or "punches out" of their company's system and goes to the supplier's web-based catalog to locate and add items to their shopping cart, while their application transparently maintains connection with the web site and gathers pertinent information. A vendor catalog, enhanced for this process, is known as a punchout catalog. PunchOut enables communication between the software and the web site so that relevant information about the transaction is delivered to the appropriate channels. 

Which in simple words, can be explained by the example of let say there is an ERP system of a large dairy plant.Now this plant may have different suppliers for all sorts of products being used in the plant e.g. Computers & related accessories, mechanical equipment, stationary. Let say the Computers are being provided by the supplier who has the punchout-ready e-commerce website. So now when dairy staff opens the procurement system (i.e SAP Ariba, Coupa etc) and there user can see the logo of the supplier, and to order the computers s/he clicks on the logo/link and redirects/punches out of the ERP  system and logs in to the supplier system.So this scenario is know as PunchOut scenario.

The following standard cXML transactions are supported by the hybris. More information about these transactions can be found at cxml.org.

Type	cXML Documents
Profile	- ProfileRequest            - ProfileResponse
PunchOut Session	- PunchOutSetupRequest            - PunchOutSetupResponse            - PunchOutOrderMessage
Purchase Order	- OrderRequest

Following diagram explains the typical sequence of this requests, here we are using Coupa as a sample procurement system,

Integration Sequence Diagram

Steps to enable the punchout feature in Hybris

I. Enable the Punchout addon and extension.

1. Ensure that b2bpunchoutaddon is listed in config/localextensions.xml and that the entry is not commented out, as shown in the following example:

   ...
   <extension name="b2bpunchoutaddon" />
   ...

2. Run the following ant task:

   ant addoninstall -Daddonnames="b2bpunchoutaddon" -DaddonStorefront.yacceleratorstorefront="yacceleratorstorefront"

3. If you renamed your storefront extension, add the following line to the config/local.properties file, as shown in the following example (substitute mystorefrontextensionname with your storefront extension name):

   ...
   [mystorefrontextensionname].additionalWebSpringConfigs.b2bpunchoutaddon=classpath:/b2bpunchoutaddon/web/spring/b2bpunchoutaddon-web-spring.xml,classpath:/b2bpunchoutaddon/web/spring/b2bpunchoutaddon-spring-security-config.xml
   ...

4. Property related to Security Channel
   Defines whether all content is served securely (https instead of http). Possible values: http, https, any.
   • When set to https, all content is served securely.
   • When set to http, most content is forced to http. Certain content like logins are still served as https.
   • When set to any, content security is not changed one way or another and is served as requested by the browser. If the main page is requested as https, for example, then all the elements of that page will be served as https. This should be recommended option.
   Default: http
   So add the following setting to config/local.properties.

   storefront.security.default.required.channel=any

5. Rebuild the system (ant all).
6. Update the system as follows:
   1. Log on the SAP Commerce Administration Console (localhost:9001).
   2. Select Platform  Update.
   3. Select the Toggle all switch (under the Project data settings section of the Update page for the newly added addon and extension).
   4. Click the Update button.

II. Set the Customers, B2BUnit and permissions.

Once the server is up and running, go to the hac, and run below impex,

#
# Import a Sample B2B Organization for Punchout
#
$passwordEncoding=md5
$defaultPassword=12341234
$setPassword=@password[translator=de.hybris.platform.impex.jalo.translators.ConvertPlaintextToEncodedUserPasswordTranslator][default='$passwordEncoding:$defaultPassword']
$b2bCustomer=B2BCustomer(uid); 

INSERT_UPDATE B2BUnit; description; uid[unique=true]; name; locName[lang=en]; groups(uid); reportingOrganization(uid); Addresses(&addId); &B2BUnitID; accountManager(uid); creditLimit(code); approvalProcessCode; 
; Hierarchical; PunchOut Organization; PunchOut Organization; PunchOut Organization;; PunchOut Organization; PunchOutAddrID; PunchOutID;;;; 
INSERT_UPDATE Address; streetname[unique=true]; postalcode[unique=true]; town; country(isocode); billingAddress; contactAddress; shippingAddress; unloadingAddress; firstname; lastname; email; title(code); &addId; owner(&B2BUnitID)[unique=true]; 
; 999 South Wacker Drive; 60606; Chicago; US; TRUE; TRUE; TRUE; TRUE; PunchOut; Customer; punchout.customer@punchoutorg.com; mr; PunchOutAddrID; PunchOutID; 

INSERT_UPDATE B2BCustomer; description; uid[unique=true]; originalUid; email; name; title(code); groups(uid); permissionGroups(uid); sessionCurrency(isocode)[default='USD']; $setPassword; 
; PunchOut Sample Customer; punchout.customer@punchoutorg.com; punchout.customer@punchoutorg.com; punchout.customer@punchoutorg.com; PunchOut Customer; mr; PunchOut Organization, b2bcustomergroup;;;;;; 
; PunchOut Sample Customer 2; punchout.customer2@punchoutorg.com; punchout.customer2@punchoutorg.com; punchout.customer2@punchoutorg.com; PunchOut Customer 2; mr; PunchOut Organization, b2bcustomergroup;;;;;; 
INSERT_UPDATE B2BCostCenter; code[unique=true]; name[lang=en]; Unit(uid); budgets(code); currency(isocode)[default='USD']
; PunchOut Organization; PunchOut Organization; PunchOut Organization;; 

INSERT_UPDATE B2BOrderThresholdTimespanPermission; code[unique=true]; Unit(uid); threshold; currency(isocode); range(code,itemtype(code));
; PunchOut Org Unlimited Timespan; PunchOut Organization; 999999999; USD; MONTH:B2BPeriodRange; 
INSERT_UPDATE B2BBudgetExceededPermission; code[unique=true]; Unit(uid); 
; PunchOut Org Budget Exceeded; PunchOut Organization;
INSERT_UPDATE B2BOrderThresholdPermission; code[unique=true]; Unit(uid); threshold; currency(isocode)[default='USD'];
; PunchOut Org Unlimited USD ORDER; PunchOut Organization; 999999999;

UPDATE B2BCustomer; originalUid[unique=true]; uid[unique=true]; permissions(code,Unit(uid)); 
; punchout.customer@punchoutorg.com; punchout.customer@punchoutorg.com; PunchOut Org Unlimited USD ORDER:PunchOut Organization,PunchOut Org Unlimited Timespan:PunchOut Organization,PunchOut Org Budget Exceeded:PunchOut Organization;
INSERT_UPDATE PunchOutCredential;code[unique=true]; domain[unique=true]; identity[unique=true]; sharedsecret
; NetworkId1; NetworkID; AN01000002779-T; VerySecret1234       
; DUNS1; DUNS; 123456789; VerySecret1234
; AribaNetworkUserId1; AribaNetworkUserId; sysadmin@ariba.com; VerySecret1234
; DUNS2; DUNS; 123; VerySecret1234
                               
INSERT_UPDATE B2BCustomerPunchOutCredentialMapping; $b2bCustomer[unique=true]; credentials(code)
; punchout.customer@punchoutorg.com; NetworkId1,DUNS1
; punchout.customer2@punchoutorg.com; AribaNetworkUserId1

Please note, this values are related to out of the box powertools shop, you may need to change this steps according to your setup.

III. Test using REST client.

Use your favorite REST client, and follow below steps to test the setup,

1. PunchOut setup request

• Request Type: Post
• URL: https://powertools.local:9002/yb2bacceleratorstorefront/punchout/cxml/setup?site=powertools
• Content-Type: application/xml
• Payload:

If the authentication succeeded, the expected response should resemble the following example.


Now, using the URL under tag, Procurement system user can automatically logged into the hybris system. Once user selects the products and go to the cart page, user will able to see the cart page like,




Once user clicks on the "RETURN TO REQUISITION" button, the cart details will be send to procurement system in the cXML fomat but in the encoded into base64. This cart will be deleted from the hybris.

2. PunchOut Order request

Once the cart/order details has been sent to the procurement system, user can either review or edit the order. After the approval process user can place an order from the procurement system. To place an order in the supplier's system (hybris storefont) below request being posted,

• Request Type: Post
• URL: https://powertools.local:9002/yb2bacceleratorstorefront/punchout/cxml/order
• Content-Type: application/xml
• Payload:

If everything goes well, you will get the response like below,



That completes the order flow.

References:

1. https://help.sap.com/viewer/4c33bf189ab9409e84e589295c36d96e/6.3.0.0/en-US/8ac40cf08669101486f5ce44920c3f91.html

2. https://success.coupa.com/Suppliers/For_Suppliers/Catalogs/Punchout_Catalogs

Blogs forever as reference points !!

These two blogs are real examples of how your blog can live forever as a reference point.

1. www.ha.ckers.org
    This blog is officially called off.It contains very interesting posts about security,very useful for web application security enthusiasts like me.Too good research work by the two great hackers.You can found very good in depth information on the topic like SQL injection,XSS and many more.And it also have  some nice DefCon pictures ;)
   
2. www.solitarygeek.com
    This blog isn't officially called off,but lastly updated on December 6th, 2010.Very good blog for novice to Java and Linux environment.Good reference point for java developers.
   
Obviously internet is full of such blogs,I found them during random surfing/searching.you can share if you found such blogs here. :)

Sixth Sense--updates.

* Sixth Sense code is now available at git : https://github.com/sixthsense/sixthsense

* There are so many activities going on at the Facebook group at : facebook.com/groups/376509655711080/

* Now,Pranav Mistry himself available to the pepole through google hangouts.You can found details about future hangouts at : plus.google.com/u/1/107376493001410152196/about

and the google meeting room : plus.google.com/hangouts/extras/talk.google.com/sixth%2520sense%2520technology?authuser=0&hl=en&eid

And the first hangouts video is available here : http://vimeo.com/sixthlab/pranav-mistry-march-15-2012