Handheld Business Solutions

Whether you have a complete specification which must simply be implemented or whether you have only a general idea of what you want, we can work with you to achieve the profitable solution that is right for your company and for your customers.

Our expertise lies in building software that runs on handheld, embedded, and mobile computers and the support software running on a server or other infrastructural computing system.

Smart card functions
Smart card standards and platforms
The advent of the cashless society
Smart card standards and platforms
Alternatives to smart cards
The shifting sands of technology
Fundamentals of card operation
Contact, contactless and combi interfaces
The challenge of interoperability
Authenticating the cardholder

Introduction

Although originally conceived in the 1970s, smart cards – as with many technologies – suffer from the use of terminology that is often imprecise and confusing. Our definitions used in the contents of this Internet site are set out in this introduction.

A smart card is a standard credit card-sized plastic token within which a microchip has been embedded. This chip is the engine room of the smart card, and indeed is what makes it ‘smart’. Smart card chips come in two broad varieties: memory-only chips, with storage space for data, and with a reasonable level of built-in security; and microprocessor chips which, in addition to memory, embody a processor controlled by a card operating system, with the ability to process data onboard, as well as carrying small programs capable of local execution. The main storage area in such cards is normally EEPROM (Electrically Erasable Programmable Read-Only Memory), which – subject to defined security constraints – can have its content updated, and which retains current contents when external power is removed. Newer smart card chips may also have maths co-processors integrated into the microprocessor chip, able to perform quite complex encryption routines relatively quickly.

A smart card is therefore characterised uniquely by its chip, with its ability to store much more data (currently up to about 32,000 bytes) than is held on a magnetic stripe, all within an extremely secure environment. These security features built into smart card chips are amongst the most sophisticated of their type available in the commercial world. Data residing in the chip can be protected against external inspection or alteration, so effectively that the vital secret keys of the cryptographic systems used to protect the integrity and privacy of card-related communications can be held safely against all but the most sophisticated forms of attack. The ingenuity of the cryptographers further supplements the physical security of the chip, ensuring that penetrating one card’s security does not compromise an entire card scheme.

It is because of these security and data storage features that smart cards are rapidly being embraced as the consumer token of choice in many areas of the public sector and commercial worlds. The Internet, in particular, is focussing the need for online identification and authentication between parties who cannot otherwise know or trust each other, and smart cards – coupled with effective cardholder verification techniques – are believed to be the most efficient and portable way of enabling the new world of e-trade. Interoperability (see below) is the key requirement to facilitate universal consumer acceptability: the ability of a card function developed by one organisation to be used without difficulty in schemes owned and operated by many organisations. So it is that the current world population of smart cards of some 1.7 billion is set to increase to 4 billion or more cards within the next 3-4 years.

[ Top ]

Smart card functions

Smart cards are being deployed in most sectors of the public and private marketplaces. Single-function cards are being used for payphone telephony, digital mobile telephony (these ‘cards’ do not in one aspect conform to the basic definition of a smart card, i.e. credit card-sized), the credit and debit functions of financial institutions, retail loyalty schemes, corporate staff systems, subscription TV operations, mass transit ticketing schemes and many more. With the advent of multi-application cards capable of carrying data relating to several functions, more complex schemes are being developed, particularly by cities for their citizens and by central Governments for their residents. In most of these schemes, simple data structures are held and updated within cards, normally comprising personal information about the cardholder and his or her accounting relationship with the card and application issuer, together with transactional data relating to the particular function. Central processing systems often mirror this data, having collected it through a polling mechanism from the terminals that accept the particular cards and enable them to participate in the related transactions.

Most smart card schemes utilise one or more generic functions, this being one of several advantages offered by smart technology. Another advantage of smart cards is that these functions are frequently associated with offline operations, i.e. functions performed without immediate access to the central system. The generic functions of cards include general transaction-based storage, storage of kernel personal data and account reference information, and – increasingly – the storage of monetary value (electronic purse) able to be loaded and spent repeatedly during the life of the card.

If, by contrast, a completely online scheme (where the user terminal can always make immediate contact with the central processing system) is implemented, the use of smart cards within such a scheme is threatened, because the data storage ability of the card might become redundant if recourse may always be made to the same data held centrally. Such permanently online schemes may be commercially viable within a single organisation, but consumer- and citizen-oriented scheme owners are increasingly recognising the benefits of issuing to the user a powerful, multi-function smart card.

The current proliferation of consumer plastic, giving rise to serious purse and wallet bulge, is focussing card issuers on the challenge of providing multi-application platforms within smart cards, able to carry functions relating not only to the card issuer’s business, but also carrying functions issued by third party application providers who may wish to rent space within such cards. This requirement has given rise to the need for suitable platforms able to carry segmented data sets in a discrete way to ensure that one application provider’s data cannot be compromised by a third party. Accordingly, a number of multi-application platform products have been developed, not only by the more traditional smart card suppliers but, more unusually, by card scheme operators with an interest in issuing cards and then defraying costs by renting space within them. Such multi-application platforms allow the addition and deletion of application data areas in-flight, without the need for replacing cards. This ability in turn leads to major branding, ownership and control issues, many of which have yet to be addressed and resolved.

[ Top ]

Smart card standards and platforms

A number of international standards bodies have concerned themselves with developing basic standards governing the physical and logical attributes of smart cards. Most of these, however, have lagged far behind the realities of technical progress, and have not addressed application level and interoperability issues sufficiently to allow the development of software to proceed with confidence. This has left space for international card players to develop products according to specifications which they severally wish to have recognised as de facto standards. Particular market sectors, which have developed such specifications, include GSM (Global System for Mobile communications) and the main credit institutions in the EMV (Europay, MasterCard, Visa) consortium. Several countries with strong central Governments are also establishing effective standards for national schemes, although Great Britain has largely stood back and allowed the fragmented development of proprietary systems to proliferate. The effect of such fragmentation will be to impose major barriers to the interoperability of cards across national and scheme boundaries (see below). In the longer term, major players with global reach – such as Microsoft – are likely to deploy cards and software in an effort to saturate the international market with a particular topography or architecture. Other possible routes to the acceptance of common standards are via a private sector consortium (Global Platform), and via the e-Europe public sector Initiative.

[ Top ]

The advent of the cashless society

In the modern world, the widespread use of cash in the form of notes and coins is increasingly being seen to hamper the effective deployment of new forms of trading. These transaction processes, as with face to face transactions, require immediate and anonymous payments. In the long-term evolution from cowry shells and tally sticks to paper and metal coinage and beyond, it is abundantly clear that the electronic storage and transfer of money value is an imperative, particularly where low-value payments require to be made internationally over telecommunications networks.

The virtual elimination of cash tokens is therefore a holy grail of national Governments, of Internet merchants and of financial institutions with a real interest in controlling and profiting from that 80% of high street expenditure currently made with notes and coins. Such electronic money can take many forms, and has been endowed with a wide and misleading vocabulary including stored value and e-purse. This has led to the development by a number of financial institutions of smart card-based products performing stored value functions, ranging from simple throw-away, burn-off cards such as payphone cards, to reloadable e-purse cards designed for low-value payments in a variety of outlets and even remotely over networks.

To date, single-function, generic e-purse cards, issued by financial institutions in various projects in a number of countries, have resulted in technical success but commercial failure in terms of usage rate. The consumer imperative and retailer benefits necessary for success have been almost completely lacking. The only truly successful examples of stored value, apart from payphone cards, have been mass transit tokens based on contactlesstechnology (see below), which have provided real convenience to consumers in cities such as Hong Kong. The race for purse is, however, just beginning, and new players are emerging to challenge the more traditional financial institutions as purse providers. The value of the pre-paid float is attractive to telcos, transport operators and retailers, and the banks are now fighting to support legislation with the aim of preventing upstarts from engaging in open purse activity. A product to watch is Sony’s Edy: an accounted purse in a contactless card – already gaining ground in Hong Kong, where it has been introduced by the Octopus mass transit card scheme (and, as a pre-requisite, the Octopus management company had to obtain a limited banking licence).

Even within the banking community, incompatible and competing purse architectures proliferate, with anonymous and non-accounted products such as Mondex (owned by MasterCard) up against fully accounted systems in the Visa camp. European initiatives, where purse schemes proliferate, are being consolidated under the Common Electronic Purse Specification (CEPS). To date, the 80+ million purse cards in Europe are still only used on average once every six weeks – a reflection on the current commercial failings of such initiatives.

[ Top ]

Alternatives to smart cards

Smart card chips are the essential operational components of smart cards, and these also appear in cladding other than credit card-sized plastic tokens. SIMs (Subscriber Identity Modules, initially implemented with simple, single application smart card chips) in a smaller physical format are already incorporated in all GSM handsets, and new developments incorporate smart chips within a variety of other devices such as PDAs and wrist watches.

There are also commercial developments seeking to place e-purse and other similar facilities within software environments in PCs and servers, obviating the need for the deployment of smart cards. It remains to be seen which of these software initiatives will flourish, particularly as it is difficult to ensure their security.

[ Top ]

The shifting sands of technology

We live in a world of fast-moving technical change. This is perhaps particularly relevant and challenging when related to smart cards, where hundreds of thousands of card-reading terminals need to be available, and tens of millions of smart cards need to be deployed, all with a potential life of several years. Forwards compatibility, and cross border and cross scheme interoperability is increasingly difficult to maintain against the background of rapid chip technology development. EEPROM may give way to faster and longer-lived Flash memory. Voltages for powering smart cards are reducing almost annually. Security technologies demand ever-faster processing power. This environment makes it extremely difficult for the confident development, acquisition and deployment of smart cards which, to support any reasonable business case, must be seen as long-term tokens.

[ Top ]

Fundamentals of card operation

Today’s smart cards need electrical power from outside, plus a way for data to be transmitted to, and read from, the chip (and in a few cases during use of the card, data is only read out, and nothing is transmitted to the chip). The cards need a timing signal (the clock) to synchronise data transmission (so that the data transmitter and receiver run at the same speed), and many microprocessor-based cards also use that timing signal to drive the microprocessor. Also, many cards (particularly contact cards – see below) have a reset line (just like the Reset button on a PC: only to be used under certain controlled circumstances if trouble is to be avoided).

[ Top ]

Contact, contactless and combi interfaces

Perhaps unfortunately, but as a result of the historical development path of this technology, there are two types of electrical interface between smart cards and their associated card readers, as follows:

Traditionally, for use at the retail point of sale or in the banking environment, or as the GSM SIM card in the mobile ‘phone, the card has a set of gold- plated electrical contacts embedded in the surface of the plastic on one side. This contact card technology is operated by inserting the card (in the correct orientation) into a slot in a card reader, which has electrical contacts that connect to the contacts on the card face.

For use in a mass transit environment, or wherever the cardholder is in motion at the moment of the transaction, radio frequency technology is used to transmit power from the reader to the card, and data is similarly transmitted over an air-gap of up to 10cms. This contactless card technology utilises an aerial coil laminated into the card, and allows communication even whilst the card is retained within a wallet or handbag. The same activation method applies to watches, pendants, baggage tags and buttons. No electrical contacts, and therefore “contactless”.

Furthermore, in more recent developments, there are now cards with both a contact and a contactless interface (dual-interface or combi-cards). These may incorporate two non-communicating chips – one for each interface – but preferably have a single, dual-interface chip providing the many advantages of a single e-purse, single operating architecture, etc.

Contactless and combi-card architectures have many advantages, but it will be several years before the main and traditional contact card-based schemes start to migrate to these technologies.

[ Top ]

The challenge of interoperability

In practice, and perhaps unfortunately for the card scheme owners and managers, different cards are usually not interchangeable. Memory cards usually have different interface characteristics from microprocessor cards: different data formats and/or electrical signals across the interface between card and terminal. Even amongst cards that appear similar, interchangeability is rare. The UK EMV bank debit/credit card scheme demands interchangeability from its various suppliers – and gets it at the level at which the cards are used by the cardholder – but this is a rare example of an attempt at interoperability. It is still uncertain whether this will extend to international emanations of EMV.

The challenge is to provide the different mixes of applications that various types of cardholder will want, while at the same time satisfying scheme and application owners. At the moment in the western hemisphere, bankers will not accept the use of a contactless interface for cash withdrawal and debit/credit payment functions. Public transport will not accept contact cards for reliability reasons, and thus the card designers have allied high data transmission speed, for high volume and cost-effective cards, only with a contactless interface. Public transport thus gets only a low security electronic purse (e-purse), suitable for closed schemes (schemes where most of the spending is directly related to the major business of the scheme owners).

[ Top ]

Authenticating the cardholder

Whilst properly designed smart cards cannot in practice be counterfeited, little progress has been made to ensure that it is the accredited cardholder who is using the genuine card. This problem is particularly acute in the e-world, where consumers are transacting business at terminals without operators able to conduct adequate verification routines.

The most common method used for cardholder verification at present is to give the cardholder a PIN (Personal Identification Number) which he or she has to remember: the cardholder has to type in the PIN at each request for signing a message, or perhaps only once per session (e.g. when the card is inserted in the card reader). PINs, however, have several disadvantages, including the risk of being stolen or abused. The only truly effective method of Cardholder Verification is the measurement of a physiological characteristic unique to an individual and incapable of fraudulent replication or abuse. Such biometrics include Iris and Retinal scans, Face or Hand geometry, and of course DNA, but the most likely and most acceptable attribute is the fingerprint. In production systems using fingerprint recognition, the fingerprint sensor is in the terminal, but the fingerprint profile data may be either in the terminal side of the card-to-terminal interface, or preferably held within the card itself (a fingerprint profile takes up only a few hundred bytes of data space). Prototype cards where the fingerprint sensor is on the card surface are now in development and may one day be a commercial proposition. In the meantime, a number of major national schemes around the world are incorporating fingerprint biometrics using optical or proximity readers associated with keyboards, mice and point-of-sale terminals.

[ Top ]

This is a write-up I did about the various reversal scenarios you can encounter on the acquiring side of a payment switch implementation.  I’ve gone through and genericized this piece a bit for distribution.  Please feel free contact me wtih any specific implementation queries.  [Note that these four Reversal scenario desingations aren't the result of any industry-specific lingo.  It's a nomenclature I invented to try to get these categorizations straight in my head and adopt a shared language with my clients and co-workers.  You may find it handy as well.]

Anyway, here’s the write-up (imagine your application in the middle taking transactions from a store-based POS device population and fowarding to a debit and/or credit gateway provider):

———————

There are four different reversal scenarios in play on any transaction acquiring environment. Each must be considered and accounted for separately. If you visualize a transaction path as starting with a customer on left, your payment switch in the middle, and the authorizing endpoint on the right, then you can think of the following “reversal points” as being presented to the reader from right to left:

· Reversal Scenario Class “R”: Remote endpoint time-out

In this scenario, the payment switch:

a) Sends original transaction request to the endpoint.

b) Does not receive response back from the authorizer prior to the pre-determined timeout period.

c) Formats and sends a ‘timeout reversal’ to the endpoint (note that some credit card-only application models do not require use of reversals of credit card-initiated transactions; consult with your authorization provider and their specs).

d) Sets the internal error code on the original transaction to “Application Route Timeout” and puts it into the tran log.

e) Formats and sends a terminal response to the store system with appropriate non-zero response code value and corresponding display message.

· Reversal Scenario Class “C”: Store System – Payment Engine Communication Rupture

In this scenario, the payment switch:

a) Sends original transaction request to the endpoint.

b) Receives a response back from the authorizer prior to the pre-determined timeout period.

c) Sets internal error code on the original transaction to the appropriate value and puts it into the tran log.

d) Formats and sends a terminal response to the store system with appropriate response code value and corresponding display message.

e) At this point in the scenario, the payment switch determines it is unable to send a reply back to the Store System origination point. The following steps should then take place:

If Internal Result Code for the original transaction is EQUAL to TRAN_APPROVED (i.e., a zero error code), then the payment switch:

1) Formats the reversal transaction that corresponds to the original.

2) Formats and sends a reversal to the endpoint (note that some credit card-only application models do not require use of reversals of credit card-initiated transactions; consult with your authorization provider and their specs).

3) Sets the internal error code on the xxxx1 reversal transaction to TRAN_APPROVED and puts it into the tran log.

4) Updates the original transaction in the tran_log by marking it as reversed.

If the payment switch internal result code for the original transaction is NOT EQUAL to TRAN_APPROVED, then no further action is required.

· Reversal Scenario Class “T”: Timeout reversal spawned by Store System

Timeout reversals are overtly generated by the point-of-sale device (or the “store system”) when the origination point does not receive a reply within a pre-specified timeout period. For the reversal process to work end-to-end, each of the authorizing endpoints specifies “match-up criteria” that allow it to locate the original on its logs.

Upon receipt of a Timeout Reversal transaction, the payment switch will execute a transaction of reversal scenario class “T.” The application must check its transaction logs to locate the corresponding original transaction. If the following conditions are true…

o Original transaction is located

o Internal Result Code = TRAN_APPROVED

o Original transaction was not already reversed

…then the payment switch will execute the following sequence of steps:

a) Formats and sends a reversal to the endpoint (note that some credit card-only application models do not require use of reversals of credit card-initiated transactions; consult with your authorization provider and their specs).

b) Sets the internal error code on the reversal transaction to TRAN_APPROVED and puts it into the tran log.

c) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

d) Updates the original transaction in the tran_log by marking it as reversed.

If the original item cannot be located on the logs, the payment switch:

a) Sets the internal error code on the reversal transaction to “Original Not Found” and puts it into the tran log.

b) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

If the original item is located, but the Internal Result Code <> TRAN_APPROVED, the payment switch:

a) Sets the internal error code on the reversal transaction to “Original Rejected” and puts it into the tran log.

b) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

If the original item is located, but was already reversed, the payment switch:

a) Sets the internal error code on the reversal transaction to “Original Already Reversed” and puts it into the tran log.

b) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

· Reversal Scenario Class “V”: In-store personnel generates Void of previous transaction

The Void is typically performed very close time-wise to its corresponding original, so for all intents and purposes it is a reversal, albeit a human-generated one.

To initiate the execution of a transaction of reversal scenario class “V,” the payment switch must check its transaction logs to locate the corresponding original transaction. If the following conditions are true…

o Original transaction is located

o Internal Result Code = TRAN_APPROVED

o Original transaction was not already reversed

…then the payment switch will execute the following sequence of steps:

a) Formats and sends a void-based reversal to the endpoint (note that some credit card-only application models do not require use of reversals of credit card-initiated transactions; consult with your authorization provider and their specs).

b) Sets the internal error code on the reversal transaction to TRAN_APPROVED and puts it into the tran log

c) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

d) Updates the original transaction in the tran_log by marking it as reversed.

If the original item cannot be located on the logs, the payment switch:

a) Sets the internal error code on the reversal transaction “Original Not Found” and puts it into the tran log.

b) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

If the original item is located, but the Internal Result Code <> TRAN_APPROVED, the payment switch:

a) Sets the internal error code on the reversal transaction “Original Rejected” and puts it into the tran log.

b) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

If the original item is located, but was already reversed, the payment switch:

a) Sets the internal error code on the reversal transaction “Original Already Reversed” and puts it into the tran log.

b) Formats and sends a terminal response to the store system with an approval response code value and corresponding display message. [NOTE: If a Reversal Class “C” scenario is encountered at this point, do not attempt to “reverse the reversal.”]

Implementing a Thales HSM 8000 Adapter – PIN Translation

In my blog, I have a lot of posts about the Thales HSM 8000 and how we implemented an adapter for it in OLS.Switch, our jPOS-based payment system.  In two recent posts, I discussed how to use jPOS’ FSDMsg facility to implement the Thales command set, and a suggestion on how to start your integration efforts – by implementing the Thales Diagnostic command (the ‘NC/ND’) as Step One.  I also have a post breaking down the command (‘CI/CJ’), the one that does “Single DES BDK to Interchange Key Translation.”  [Make sure you read through the comments on that post, too.] 

Recently, we were asked to augment our PIN Translation capabilities in order to support a population of PIN Pads that generate PIN blocks using either Single and Triple DES BDKs.  I mentioned briefly in other posts that there’s a separate command to do “Triple DES BDK to Interchange Key Translation” – the G0/G1.  In this particular situation, the devices could only support double-length keys (Triple-length keys are an option when working with Triple DES BDKs), so our keys configuration file was going to have all double-length BDKs.  We couldn’t use cryptogram length to signal what’s what to our program, so we needed some type of indicator.  So, we came up with the idea of these ’1des’ and ’3des’ tags in the file to act as identifiers, like this (where ’129908′ and ’184402′ are examples of KSIs – the first six positions of the KSN in this particular implementation…read that Single DES BDK…” post referenced above for more details):

bdk.129908=1des:99BF6A880305B37775AE1E129676E421
bdk.184402=1des:80B7F397139F99A01266394CA96384D9

Then, we have our translatePinblk code in our adapter.  We modified it to handle both key types and generate ‘CI’ or ‘G0′ commands (for Single or Triple DES BDK Translation respectively).  Behind the scenes, what makes this code work is the jPOS FSDMsg facility.  We’ve got the ‘CI’ and ‘G0′ structures defined, and that bolded line there fills in the key value (the ‘command’ field).  You can see that the ‘G0′ has one extra field in its message format:

    public FSDMsg translatePinblk (
        String bdk, String zpk, String ksnDescriptor,
        String ksn, String pinblk, String accountNumber)
    {
        boolean threeDesBdk = false;
        if (bdk.startsWith (“3des:”)) {
            threeDesBdk = true;
            bdk = bdk.substring(5);
        } else if (bdk.startsWith (“1des:”)) {
            bdk = bdk.substring(5);
        }
        FSDMsg r = createRequest (threeDesBdk ? “G0″ : “CI”);
        r.set (“bdk-type”, “U”);
        r.set (“bdk”, bdk);
        r.set (“zpk-type”, “U”);
        r.set (“zpk”, zpk);
        r.set (“ksn-descriptor”, ksnDescriptor);
        r.set (“ksn”, ksn);
        r.set (“pinblk”, pinblk);
        r.set (“destination-pinblk-format”, “01″);  // ANSI
        r.set (“account-number”, accountNumber);
        if (threeDesBdk)
            r.set (“source-pinblk-format”, “01″); 

        return command (r);
    }

Now, here are some resulting traces ; I turned on tracing to the test HSM, so you can see all six legs (not presented in order, but you’ll get the idea):

• Request from the device
• Request to the Thales (The Single DES example is first, followed by the Triple DES example).
• Response from the Thales
• Request to the Debit/EBT Gateway (the Translated PIN block goes into Field 52 on the outgoing message – the ‘FFFFFF’ in the middle is a mask applied for tracing purposes only).
• Response from the Debit/EBT Gateway
• Response to the device

<log realm=”org.jpos.security.thales.ThalesAdapter” at=”Tue Apr 24 16:09:45 EDT 2007.378″>
  <trace>
    <fsdmsg schema=’file:cfg/hsm-base’>
      command: ‘CI’
      bdk: ’3A796910699174AEEF512C5A489AB008′
      zpk-type: ‘U’
      zpk: ‘C6087BAD7B9827F553DDF858709E7030′
      ksn-descriptor: ’605′
      ksn: ’0407030000200016′
      pinblk: ’32D23493EC13C57F’
      destination-pinblk-format: ’01′
      account-number: ’777999999901′
    </fsdmsg>
     request: ‘CI3A796910699174AEEF512C5A489AB008UC6087BAD7B9827F553DDF
               858709E7030605040703000020001632D23493EC13C57F01777999999901′
    response: ‘CJ0004D354D87634C043FE01′
     elapsed: 156ms
    <fsdmsg schema=’file:cfg/hsm-resp-base’>
      response: ‘CJ’
      error: ’00′
      pin-length: ’04′
      pinblk: ‘D354D87634C043FE’
    </fsdmsg>
  </trace>
</log>
<log realm=”Debug” at=”Tue Apr 24 16:09:45 EDT 2007.909″>
  <commit>
    <id>9</id>
    <context>
     <transient>
      <entry key=”PROFILER”>
       <profiler>
         open [0/0]
         parse-request [0/0]
         create-debit-tranlog [15/15]
         populate-debit-tranlog [16/31]
         validate-terminal [16/47]
         find-duplicate [15/62]
         translate-pin [172/234]
         create-fdr-request [16/250]
         query-host-or-reverse [453/703]
         prepare-debit-response [0/703]
         close [62/765]
         send-response [0/765]
         end [765/765]
       </profiler>
      </entry>
      <entry key=”DB”>org.jpos.ee.DB@e47e9b</entry>
      <entry key=”REQUEST”>
       <fsdmsg schema=’file:cfg/v2-base’>
         header: ’94538660′
         record-format: ‘G’
         application-type: ’0′
         field-sep-1: ‘.’
         acquirer-bin: ’628702′
         message-version: ’02′
         shift-number: ’00006′
         merchant-number: ’0010′
         store-number: ’01856′
         terminal-number: ’0001′
         category-code: ’4444′
         country-code: ’987′
         city-code: ’97208′
         timezone-differential: ’010′
         transaction-code: ’93′
         terminal-serial-number: ’12345678′
         encryption-indicator: ’3′
         transaction-sequence-number: ’2480′
         card-id-source: ‘A’
         account-entry-mode: ‘D’
         magnetic-strip-info: ’401777______9011=_________________’
         cid-device-attached: ‘X’
         pin-action-code: ‘F’
         pin-function-code: ’0′
         pin-length: ’00′
         pin-block: ’00′
         encrypted-pin-block: ’32D234______C57F’
         pin-dukpt-ksn: ’040703______0016′
         amount: ’322′
         additional-amount: ’0′
         register-number: ’04′
         tran-id: ’0000276′
         tender-attempt-indicator: ’1′
         tender-number: ’0001′
         tender-attempt: ’0203′
       </fsdmsg>
      </entry>
      <entry key=”switch-key”>G.93</entry>
      <entry
       key=”TRANLOG”>org.jpos.ee.DebitTranLog@1b98775[id=1987442]</entry>
      <entry key=”AUDIT”>true</entry>
      <entry key=”MERCHANT”>org.jpos.ee.Merchant@46e3f0[id=0010]</entry>
      <entry key=”TERMINAL”>org.jpos.ee.Terminal@1b8be78[id=1]</entry>
      <entry key=”STORE”>org.jpos.ee.Store@72ca69[id=01856]</entry>
      <entry key=”RESPONSE”>
       <fsdmsg schema=’file:cfg/v2-debit-resp-base’>
         header: ’94538660′
         record-format: ’0′
         terminal-serial-number: ’12345678′
         encryption-indicator: ’3′
         transaction-sequence-number: ’2480′
         magnetic-strip-info: ’401777______9011=_________________’
         register-number: ’04′
         tran-id: ’0000276′
         tender-attempt-indicator: ’1′
         tender-number: ’0001′
         tender-attempt: ’0203′
         response-code: ‘AA’
         display-message: ‘APPROVAL’
         approval-number: ’210220′
         date: ’042407′
         capture-date: ’042407′
       </fsdmsg>
      </entry>
     </transient>
     <persistent>
      <entry key=”TIMESTAMP”>Tue Apr 24 16:09:45 EDT 2007</entry>
      <entry key=”OUTSTANDING”>0</entry>
      <entry key=”TRANLOG_ID”>1987442</entry>
      <entry key=”CAPTURE_DATE”>Tue Apr 24 00:00:00 EDT 2007</entry>
      <entry key=”ISO_REQUEST”>
       <isomsg direction=”outgoing”>
         <field id=”0″ value=”0200″/>
         <field id=”3″ value=”009000″/>
         <field id=”4″ value=”322″/>
         <field id=”7″ value=”0424200945″/>
         <field id=”11″ value=”003716″/>
         <field id=”12″ value=”160945″/>
         <field id=”13″ value=”0424″/>
         <field id=”18″ value=”5912″/>
         <field id=”22″ value=”901″/>
         <field id=”24″ value=”001″/>
         <field id=”25″ value=”00″/>
         <field id=”35″ value=”401777______9011=_________________”/>
         <field id=”37″ value=”711401856480″/>
         <field id=”41″ value=”00389999″/>
         <field id=”42″ value=”000452002888888″/>
         <field id=”49″ value=”840″/>
         <field id=”52″ value=”D354D8FFFFFF43FE” type=”binary”/>
       </isomsg>
      </entry>
      <entry key=”ISO_RESPONSE”>
       <isomsg direction=”incoming”>
         <field id=”0″ value=”0210″/>
         <field id=”3″ value=”009000″/>
         <field id=”4″ value=”000000000322″/>
         <field id=”7″ value=”0424200945″/>
         <field id=”11″ value=”003716″/>
         <field id=”12″ value=”160945″/>
         <field id=”13″ value=”0424″/>
         <field id=”24″ value=”001″/>
         <field id=”25″ value=”00″/>
         <field id=”37″ value=”711401856480″/>
         <field id=”38″ value=”210220″/>
         <field id=”39″ value=”00″/>
         <field id=”41″ value=”00389999“/>
         <field id=”63″ value=”415050524F56414C2020202020202020″ type=”binary”/>
       </isomsg>
      </entry>
      <entry key=”RC”>0000</entry>
     </persistent>
    </context>
  </commit>
</log>

<log realm=”org.jpos.security.thales.ThalesAdapter” at=”Wed Apr 25 09:49:55 EDT 2007.56″>
  <trace>
    <fsdmsg schema=’file:cfg/hsm-base’>
      command: ‘G0′
      bdk: ‘D76FCCC0AB5F4A74875B39B9ADE6E900′
      zpk-type: ‘U’
      zpk: ‘B381A477B49A67C9581DFC81E9CA404C’
      ksn-descriptor: ’605′
      ksn: ’3D07040000000003′
      pinblk: ’661C4571C8A88998′
      destination-pinblk-format: ’01′
      account-number: ’765111111111′
      source-pinblk-format: ’01′
    </fsdmsg>
     request: ‘G0D76FCCC0AB5F4A74875B39B9ADE6E900UB381A477B49A67C9581DFC81E9
               CA404C6053D07040000000003661C4571C8A889980101765111111111′
    response: ‘G1000449FDE47BE6B24C1F01′
     elapsed: 94ms
    <fsdmsg schema=’file:cfg/hsm-resp-base’>
      response: ‘G1′
      error: ’00′
      pin-length: ’04′
      pinblk: ’49FDE47BE6B24C1F’
    </fsdmsg>
  </trace>
</log>
<log realm=”Debug” at=”Wed Apr 25 09:49:55 EDT 2007.369″>
  <commit>
    <id>12</id>
    <context>
     <transient>
      <entry key=”PROFILER”>
       <profiler>
         open [0/0]
         parse-request [0/0]
         create-debit-tranlog [16/16]
         populate-debit-tranlog [15/31]
         validate-terminal [16/47]
         find-duplicate [15/62]
         translate-pin [94/156]
         create-fdr-request [16/172]
         query-host-or-reverse [281/453]
         prepare-debit-response [0/453]
         close [16/469]
         send-response [0/469]
         end [469/469]
       </profiler>
      </entry>
      <entry key=”DB”>org.jpos.ee.DB@10457b4</entry>
      <entry key=”REQUEST”>
       <fsdmsg schema=’file:cfg/v2-base’>
         header: ’3F0FD3C0′
         record-format: ‘G’
         application-type: ’0′
         field-sep-1: ‘.’
         acquirer-bin: ’628702′
         message-version: ’02′
         shift-number: ’00006′
         merchant-number: ’0010′
         store-number: ’01856′
         terminal-number: ’0001′
         category-code: ’4444′
         country-code: ’987′
         city-code: ’97208′
         timezone-differential: ’010′
         transaction-code: ’93′
         terminal-serial-number: ’12345678′
         encryption-indicator: ’3′
         transaction-sequence-number: ’2500′
         card-id-source: ‘A’
         account-entry-mode: ‘D’
         magnetic-strip-info: ’421765______1119=____________________’
         cid-device-attached: ‘X’
         pin-action-code: ‘F’
         pin-function-code: ’0′
         pin-length: ’00′
         pin-block: ’00′
         encrypted-pin-block: ’661C45______8998′
         pin-dukpt-ksn: ’3D0704______0003′
         amount: ’2500′
         additional-amount: ’2000′
         register-number: ’04′
         tran-id: ’0000300′
         tender-attempt-indicator: ’1′
         tender-number: ’0003′
         tender-attempt: ’0203′
         magnetic-strip-info-encrypted: ‘kOGgk2__________________IsD+’
       </fsdmsg>
      </entry>
      <entry key=”switch-key”>G.93</entry>
      <entry   
       key=”TRANLOG”>org.jpos.ee.DebitTranLog@1fc68f[id=1987462]</entry>
      <entry key=”AUDIT”>true</entry>
      <entry key=”MERCHANT”>org.jpos.ee.Merchant@1cd36a5[id=0010]</entry>
      <entry key=”TERMINAL”>org.jpos.ee.Terminal@1c914c3[id=1]</entry>
      <entry key=”STORE”>org.jpos.ee.Store@2e0ecb[id=01856]</entry>
      <entry key=”RESPONSE”>
       <fsdmsg schema=’file:cfg/v2-debit-resp-base’>
         header: ’3F0FD3C0′
         record-format: ’0′
         terminal-serial-number: ’12345678′
         encryption-indicator: ’3′
         transaction-sequence-number: ’2500′
         magnetic-strip-info: ’421765______1119=____________________’
         register-number: ’04′
         tran-id: ’0000300′
         tender-attempt-indicator: ’1′
         tender-number: ’0003′
         tender-attempt: ’0203′
         response-code: ‘AA’
         display-message: ‘APPROVAL’
         approval-number: ’312521′
         date: ’042507′
         capture-date: ’042507′
       </fsdmsg>
      </entry>
     </transient>
     <persistent>
      <entry key=”TIMESTAMP”>Wed Apr 25 09:49:54 EDT 2007</entry>
      <entry key=”OUTSTANDING”>0</entry>
      <entry key=”TRANLOG_ID”>1987462</entry>
      <entry key=”CAPTURE_DATE”>Wed Apr 25 00:00:00 EDT 2007</entry>
      <entry key=”ISO_REQUEST”>
       <isomsg direction=”outgoing”>
         <field id=”0″ value=”0200″/>
         <field id=”3″ value=”099000″/>
         <field id=”4″ value=”2500″/>
         <field id=”7″ value=”0425134954″/>
         <field id=”11″ value=”003730″/>
         <field id=”12″ value=”094954″/>
         <field id=”13″ value=”0425″/>
         <field id=”18″ value=”5912″/>
         <field id=”22″ value=”901″/>
         <field id=”24″ value=”001″/>
         <field id=”25″ value=”00″/>
         <field id=”35″ value=”421765______1119=____________________”/>
         <field id=”37″ value=”711501856500″/>
         <field id=”41″ value=”00389999“/>
         <field id=”42″ value=”000452002888888″/>
         <field id=”49″ value=”840″/>
         <field id=”52″ value=”49FDE4FFFFFF4C1F” type=”binary”/>
         <field id=”54″ value=”2000″/>
       </isomsg>
      </entry>
      <entry key=”ISO_RESPONSE”>
       <isomsg direction=”incoming”>
         <field id=”0″ value=”0210″/>
         <field id=”3″ value=”099000″/>
         <field id=”4″ value=”000000002500″/>
         <field id=”7″ value=”0425135031″/>
         <field id=”11″ value=”003730″/>
         <field id=”12″ value=”094954″/>
         <field id=”13″ value=”0425″/>
         <field id=”24″ value=”001″/>
         <field id=”25″ value=”00″/>
         <field id=”37″ value=”711501856500″/>
         <field id=”38″ value=”312521″/>
         <field id=”39″ value=”00″/>
         <field id=”41″ value=”00389999“/>
         <field id=”63″ value=”415050524F56414C2020202020202020″ type=”binary”/>
       </isomsg>
      </entry>
      <entry key=”RC”>0000</entry>
     </persistent>
    </context>
  </commit>
</log>

I’ve had a number of people ask me recently about how to implement Dynamic Key Exchange models within jPOS.   Specifically, I’m talking here about ISO8583-based financial payment gateways.  This post pertains to situations where you’re acting either as the Card Issuer (in which case you’re receiving payment transaction requests from the gateway) or the transaction acquirer (in which case you’re sending payment transaction requests to the gateway in order that they route it for appropriate authorization decisioning).

There’s some terminology to square away first:

Local Master Key (‘LMK’) – This is the key you store in the HSM in order to encrypt and do software-based storage of the current Working Keys (and Base Derivation Keys if you’re using DUKPT).  Also called the Master File Key (‘MFK’)

Zone PIN Key (‘ZPK’) – The ZPK is what’s used to encrypt the PIN blocks that traverse the wires between institutions.  Also referred to as the ‘Working Key.’  This is the key that the Dynamic Key Exchange is acting upon.  You’re obligated to change the Working Key at agreed-upon intervals (I typically advocate every 12 hours).

Zone Master Key (‘ZMK’) -  Think of the ZMK as the key transportation vehicle.  It’s the key that the two parties use to encrypt and exchange new ZPKs.  This key is established via a key ceremony.  You keep a copy of the ZMK encrypted under the LMK in a file somewhere (you’ll see how it’s used here further down this post).  Also called the Key Exchange Key (‘KEK’).

Now, I’ll pass along some hard-earned ‘lessons learned’ (Alejandro and I have the scars on our backs to prove it):

1. From the moment you start planning discussions with the gateway, establish RIGHT AWAY that you want field-by-field level specifics of how the Dynamic Key Exchange is to be performed.  It’ll be within the context of some  Network Message Exchange (e.g., 0800/0810), but that’s not granular enough – you need to know the thing down to the field-content level.
2. Scour the documentation you’ve been provided to see if those details are in there.  I’ve done two different gateway projects recently, and in both cases the Key Exchange details were notably absent from the doc.  But, that doc exists somewhere within the gateway institution.  Track it down.  Get your hands on it. 
3. Knowledge of the Key Exchange model is – by design – not widespread throughout the gateway provider’s project personnel.  Insist on getting their expert in on at least one of the planning calls.  Make note of this person’s name and contact details.  Establish that information channel.  This is a critically important point to your success.

At a high level, there are two models:

• You request a new ZPK from the gateway, and they provide it in the response.  [I call this the 'Pull' model (for obvious reasons - you pull the key from them).]
• The gateway sends you a new ZPK and you respond with a message indicating success or failure.  [This, by contrast is the 'Push' model.]

Your implementation will be one of those.

Now, I’ll provide two examples, one push, one pull.  In both cases, I’ll reference some files that I prepared that will show you the model explained as implemented in fine detail. 

Here’s the push model.  [I've done some annotating there within Acrobat.]  We’re the Issuer in this example.  The gateway sends us a new key every 12 hours.  Behind the scenes in this example we’re using a Key-Up II box from IDS as our HSM.   The sequence of events is:

1. The gateway sends us a new ZPK (under ZMK) in an 0800 (MTI) Network Request.
2. We obtain the ZMK (under LMK) from our files.
3. We use the cryptograms from Steps 1 and 2 to create the appropriate command to the Key-Up (here, a ’12′)
4. We get the response from the Key-Up (the ’13′) and validate that the Check Digits match those provided by the Issuer.
5. Assuming the check in Step 4 is okay, we store the result (the ZPK under LMK) as the new Working Key.
6. We send an 0810 (MTI) Network Response back to the Issuer (Note that Field 39 on our response is ’00′ – indicating success). 

There’s so much detail here worthy of comment.  I’ll touch on a few things (these are the types of detail you want to bring to the surface in your reviews):

• This gateway uses ’162′ in Field 70 to tip to us that it’s a Key Exchange in play.
• Note how we have to pluck the incoming cryptogram out of the esoteric morass of Field 123.
• We have to construct an equally cryptic Field 123 on our response.

For reference – the push example provided here is a working model for Genpass (now called Elan Financial Services – a division of US Bank) interface.  Nice people!

Here’s the pull model.  [Here, I had to do a did a bit of redacting in Acrobat.  The redactions don't spoil the flow.]  I won’t go into too much detail in-line here, because you’ll see I’ve provided entirely TMI (Too Much Information!) in the file.  The flow here is:

1. We request a new key from the Gateway in an 0800.
2. The new key (ZPK under ZMK) comes back in an 0810.
3. We fire off an ‘FA’ to the Thales 8000.
4. We get the ‘FB’ back and validate the check digits.
5. If okay, we store the result (the ZPK under LMK) as the new Working Key.

Now, since we’re the initiator here we have to have a way to determine when to trigger the exchange request.  We do that through a channel Logon Manager.  Here’s a 30_fdr_logon_mgr.xml we have defined:

<fdr-logon-mgr class=”org.jpos.fdr.LogonManager” logger=”Q2″>
<property name=”persistent-space” value=”jdbm:fdrlogon:log/fdrlogon” />
<property name=”mux”            value=”fdr-mux” />
<property name=”channel-ready”  value=”fdr.ready” />
<property name=”timeout”        value=”900000″ />
<property name=”echo-interval”  value=”600000″ />
<property name=”logon-interval” value=”43200000″ />
<property name=”pseudo-tid”     value=”00599999″ />
<property name=”pseudo-mid”     value=”000452009999999″ />
<property name=”sm”             value=”Thales” />
<property file=”cfg/keys.cfg” />
<property name=”initial-delay”  value=”5000″ />
</fdr-logon-mgr>

You can see the ‘logon-interval’  – 43,200,000 milliseconds is every 12 hours  (behind the scenes, our logon sequence does a logon exchange and a key exchange.]

For reference, the pull example provided here is a working model for the FDR North interface.

You get the idea, I hope!  Nail down all those details in order to maximize your chances of success.  Otherwise, feel free to beat your head against a wall, because that’s what will happen if you don’t get this information.

This page explains common techniques to deal with the Session and transactions in Hibernate applications. Refer to the Hibernate reference documentation and the “Transactions and Concurrency” chapter for more information. This page describes Hibernate 3.1.x and code shown here does not work in older versions.

• Unit of Work
• Transactions
• The scope of a unit of work
• Transaction demarcation with JTA
• Transaction demarcation with plain JDBC
• Transaction demarcation with EJB/CMT
• Custom transaction interceptors
• Implementing long Conversations
• Implementing data access objects (DAOs)
• What about the SessionFactory?
• This is all very difficult, can’t this be done easier?

Unit of Work

A particular unit of work is grouping data access operations. We usually refer to the Hibernate Session as a unit of work because the scope of a Session is exactly that, in almost all cases. (The Session is also many other things, for example, a cache and a primary API.) To begin a unit of work you open a Session. To end a unit of work you close a Session. Usually you also flush a Session at the end of a unit of work to execute the SQL DML operations (UPDATE, INSERT, DELETE) that synchronize the in-memory Session state with the database. A Session executes also SQL queries, whenever the developer triggers a query with the API or through loading on demand (lazy loading). Alternatively, think of the Session as a gateway to your database, a map of managed entity instances that are automatically dirty checked, and a queue of SQL DML statements that are created and flushed by Hibernate automatically.

Transactions

Transactions also group data access operations, in fact, every SQL statement, be it queries or DML, has to execute inside a database transaction. There can be no communication with a database outside of a database transaction. (Note that there are such things as read-only transactions, that can be used to improve cleanup time in a database engine if it is not smart enough to optimize its own operations.)

One approach is the auto-commit mode, where every single SQL statement is wrapped in a very short transaction. This mode is never appropriate for an application, but only for ad-hoc execution of SQL with an operator console. Hibernate disables or expects the environment (in J2EE/JEE) to disable auto-commit mode, as applications are not executing ad-hoc SQL but a planned sequence of statements. (There are ways to enable auto-commit behavior in Hibernate but it is by definition slower than regular transactions and less safe. If you want to know more about auto-commit mode, read this.)

The right approach is to define clear transaction boundaries in your application by beginning and committing transactions either programmatically, or if you have the machinery to do this, declaratively (e.g. on service/command methods). If an exception occurs the transaction has to be rolled back (or declaratively, is rolled back).

The scope of a unit of work

A single Hibernate Session might have the same scope as a single database transaction.

This is the most common programming model used for the session-per-request implementation pattern. A single Session and a single database transaction implement the processing of a particular request event (for example, a Http request in a web application). Do never use the session-per-operation anti-pattern! (There are extremely rare exceptions when session-per-operation might be appropriate, you will not encounter these if you are just learning Hibernate.)

Another programming model is that of long Conversations, e.g. an application that implements a multi-step dialog, for example a wizard dialog, to interact with the user in several request/response cycles.

One way to implement this is the session-per-request-with-detached-objects pattern. Once persistent objects are considered detached during user think-time and have to be reattached to a new Session after they have been modified.

The session-per-conversation pattern is however recommended. In this case a single Session has a bigger scope than a single database transaction and it might span several database transactions. Each request event is processed in a single database transaction, but flushing of the Session would be delayed until the end of the conversation and the last database transaction, to make the conversation atomic. The Session is held in disconnected state, with no open database connection, during user think-time. Hibernate’s automatic optimistic concurrency control (with versioning) is used to provide conversation isolation.

Hibernate supports several convenience APIs that make implementation of all transaction and conversation strategies easier, with any transaction processing system you might deploy on.

Transaction demarcation with JTA

Hibernate works in any environment that uses JTA, in fact, we recommend to use JTA whenever possible as it is the standard Java transaction interface. You get JTA built-in with all J2EE/JEE application servers, and each Datasource you use in such a container is automatically handled by a JTA TransactionManager. But this is not the only way to get JTA, you can use a standalone implementation (e.g. JOTM) in any plain JSE environment. Another example is JBoss Seam, it comes bundled with a demo application that uses an embeddable version of the JBoss JCA/JTA/JNDI services, hence provides JTA in any deployment situation.

Hibernate can automatically bind the “current” Session to the current JTA transaction. This enables an easy implementation of the session-per-request strategy with the getCurrentSession() method on your SessionFactory:

try {     UserTransaction tx = (UserTransaction)new InitialContext()                             .lookup("java:comp/UserTransaction");                                  tx.begin();      // Do some work     factory.getCurrentSession().load(...);     factory.getCurrentSession().persist(...);      tx.commit(); } catch (RuntimeException e) {     tx.rollback();     throw e; // or display error message }

The advantage of the built-in support should become clear as soon as you write non-trivial applications: you can separate the transaction demarcation code from your data access code. The “current session” refers to a Hibernate Session bound by Hibernate behind the scenes, to the transaction scope. A Session is opened when getCurrentSession() is called for the first time and closed when the transaction ends. It is also flushed automatically before the transaction commits. You can call getCurrentSession() as often and anywhere you want as long as the transaction runs. To enable this strategy in your Hibernate configuration:

• set hibernate.transaction.manager_lookup_class to a lookup strategy for your JEE container
• set hibernate.transaction.factory_class to org.hibernate.transaction.JTATransactionFactory

See the Hibernate reference documentation for more configuration details.

This does not mean that all Hibernate Sessions are closed when a transaction is committed! Only the Session that you obtained with sf.getCurrentSession() is flushed and closed automatically. If you decide to use sf.openSession() and manage the Session yourself, you have to flush() and close() it. So a less convenient alternative, without any “current” Session, is this:

UserTransaction tx = (UserTransaction)new InitialContext()                             .lookup("java:comp/UserTransaction");  Session session = factory.openSession();  try {     tx.begin();      // Do some work     session.load(...);     session.persist(...);      session.flush();      tx.commit(); } catch (RuntimeException e) {     tx.rollback();     throw e; // or display error message } finally {     session.close(); }

If you manage the Session yourself, code is more difficult to layer. You can’t easily move data access operations into a different layer than transaction and Session demarcation.

Transaction demarcation with plain JDBC

If you don’t have JTA and don’t want to deploy it along with your application, you will usually have to fall back to JDBC transaction demarcation. Instead of calling the JDBC API you better use Hibernate’s Transaction and the built-in session-per-request functionality:

try {     factory.getCurrentSession().beginTransaction();      // Do some work     factory.getCurrentSession().load(...);     factory.getCurrentSession().persist(...);      factory.getCurrentSession().getTransaction().commit(); } catch (RuntimeException e) {     factory.getCurrentSession().getTransaction().rollback();     throw e; // or display error message }

Because Hibernate can’t bind the “current session” to a transaction, as it does in a JTA environment, it binds it to the current Java thread. It is opened when getCurrentSession() is called for the first time, but in a “proxied” state that doesn’t allow you to do anything except start a transaction. When the transaction ends, either through commit or roll back, the “current” Session is closed automatically. The next call to getCurrentSession() starts a new proxied Session, and so on. In other words, the session is bound to the thread behind the scenes, but scoped to a transaction, just like in a JTA environment. This thread-bound strategy works in every JSE application – note that you should use JTA and a transaction-bound strategy in a JEE environment (or install JTA with your JSE application, this is a modular service).

To enable the thread-bound strategy in your Hibernate configuration:

• set hibernate.transaction.factory_class to org.hibernate.transaction.JDBCTransactionFactory
• set hibernate.current_session_context_class to thread

This does not mean that all Hibernate Sessions are closed when a transaction is committed! Only the Session that you obtained with sf.getCurrentSession() is flushed and closed automatically. If you decide to use sf.openSession() and manage the Session yourself, you have to close() it. So a less convenient alternative, without any “current” Session, is this:

Session session = factory.openSession(); Transaction tx = null; try {     tx = session.beginTransaction();      // Do some work     session.load(...);     session.persist(...);      tx.commit(); // Flush happens automatically } catch (RuntimeException e) {     tx.rollback();     throw e; // or display error message } finally {     session.close(); }

If you manage the Session yourself, code is more difficult to layer. You can’t easily move data access operations into a different layer than transaction and Session demarcation.

Transaction demarcation with EJB/CMT

Our goal really is to remove any transaction demarcation code from the data access code:

@TransactionAttribute(TransactionAttributeType.REQUIRED) public void doSomeWork() {     // Do some work     factory.getCurrentSession().load(...);     factory.getCurrentSession().persist(...); }

Instead of coding the begin, commit, and rollback of your transactions into your application you could use a declarative approach. For example, you might declare that some of your service or command methods require a database transaction to be started when they are called. The transaction ends when the method returns; if an exception is thrown, the transaction will be rolled back. The Hibernate “current” Session has the some scope as the transaction (flushed and closed at commit) and is internally also bound to the transaction. It propagates into all components that are called in one transactions.

Declarative transaction demarcation is a standard feature of EJB, also known as container-managed transactions (CMT). In EJB 2.x you would use XML deployment descriptors to create your transaction assembly. In EJB 3.x you can use JDK 5.0 annotation metadata directly in your source code, a much less verbose approach. To enable CMT transaction demarcation for EJBs in Hibernate configuration:

• set hibernate.transaction.manager_lookup_class to a lookup strategy for your JEE container
• set hibernate.transaction.factory_class to org.hibernate.transaction.CMTTransactionFactory

Custom transaction interceptors

To remove transaction demarcation from your data access code you might want to write your own interceptor that can begin and end a transaction programmatically (or even declaratively). This is a lot easier than it sounds, after all, you only have to move three methods into a different piece of code that runs every time a request has to be processed. Of course more sophisticated solutions would also need to handle transaction propagation, e.g. if one service method calls another one. Typical interceptors are a servlet filter, or an AOP interceptor that can be applied to any Java method or class.

For an implementation with a servlet filter see Open Session in View.

For an implementation with JBoss AOP see Session handling with AOP.

Implementing long Conversations

If you’d like to design your application with a session-per-conversation strategy, you need to manage the “current” Session yourself. An example with a servlet filter is shown with the Open Session in View pattern.

Implementing data access objects (DAOs)

Writing DAOs that call Hibernate is incredibly easy and trivial. You don’t need a framework. You don’t need to extend some “DAOSupport” superclass from a proprietary library. All you need to do is keep your transaction demarcation (begin and commit) as well as any Session handling code outside of the DAO implementation. For example, a ProductDAO class has a setCurrentSession() method or constructor, or it looks up the “current” Hibernate Session internally. Where this current Session comes from is not the responsibility of the DAO! How a transaction begins and ends is not the responsibility of the DAO! All the data access object does is use the current Session to execute some persistence and query operations. For a pattern that follows these rules, see Generic Data Access Objects.

What about the SessionFactory?

In the examples above you can see access to the SessionFactory. How do you get access to the factory everywhere in your code? Again, if you run in a JEE environment, or use an embedded service in JSE, you could simply look it up from JNDI, where Hibernate can bind it on startup. Another solution is to keep it in a global static singleton after startup. You can in fact solve both the problem of SessionFactory lookup and Hibernate startup with the same piece of code, a trivial helper class (this is from the tutorial in chapter 1, Hibernate reference documentation):

public class HibernateUtil {      private static final SessionFactory sessionFactory;      static {         try {             // Create the SessionFactory from hibernate.cfg.xml             sessionFactory = new Configuration().configure().buildSessionFactory();         } catch (Throwable ex) {             // Make sure you log the exception, as it might be swallowed             System.err.println("Initial SessionFactory creation failed." + ex);             throw new ExceptionInInitializerError(ex);         }     }      public static SessionFactory getSessionFactory() {         return sessionFactory;     }  }

A more sophisticated version if HibernateUtil that can also switch automatically between JNDI and static singleton can be found in the CaveatEmptor demo application.

Note: There are many variations of much more complex HibernateUtil classes floating around the net. However, for Hibernate 3.1, the code shown above is the only code that is needed. If you use JNDI you might want to have a look at HibernateUtil in the latest CaveatEmptor. Every other HibernateUtil is obsolete for Hibernate 3.1.

This is all very difficult, can’t this be done easier?

Hibernate can only do so much as a persistence service, managing the persistence service is however the responsibility of the application infrastructure, or framework. The EJB3 programming model makes transaction and persistence context management very easy, use the Hibernate EntityManager to get this API. Either run your EJBs inside a full J2EE application server (previews available from several vendors) or in a lightweight embeddable EJB3 container, JBoss Embeddable EJB3, in any Java environment. The JBoss Seam framework has built-in support for automatic context management, including persistence and conversations, with only a few annotations in your source code.

In legacy payment systems, the ‘databases’ employed tend to be esoteric, closed, proprietary subsystems hard-wired into the application itself.  For example, in ON/2 (running on the Stratus VOS platform) the database is an internal sub-system called ‘DBI.’  This makes sense: when ON/2 was first rolled out (1982!), SQL was in its infancy (I think Larry Ellison was still on his first wife) and OLTP vendors were ‘rolling their own’ to meet very specific database needs.

And let’s be clear here:  those internal inventions have stood the test of time.  ‘dbi’ is approaching 25 years and is still serviceable.  There are some serious shortcomings to the approach though:

• The transaction log – nominally a database table – is, in fact, just a log.  This fact greatly and unnecessarily complicates (note: this is a personal opinion I’m expressing here) some of the very critical touchpoints of your financial switch: namely, how you implement ‘multi-part’ transactions like reversals and completions (of pre-auths).
• It’s very esoteric and akin to a black art.   The great thing about the SQL world is that everything is standards-based.  Our clients have deep bench strength in the SQL vendor offering they’ve selected (e.g., Oracle, MS SQL Server, etc.) and they can contribute all of their expertise withough having to have internal knowledge of the jPOS application or even a general knowledge of payment systems.  This is huge.  A talented DBA is a very important part of a successful jPOS implementation.  I know this from experience.
• It completely wipes out any flexbility you have in terms of dropping that application (as is) onto another operating platform.  Continuing with the ‘dbi’ example, that’s a Stratus VOS construct pure and simple.  You’re tied to that thing like a boat anchor.  With a SQL-based approach, you’ve got flexbility.  Assuming your app is written in Java and ‘talks’ to a SQL DB via JDBC (that accurately describes jPOS), then you can be (as an example) Windows/SQL Server today and Linux/MySQL next week (we’ve actually done it on the same day).  That’s a pipe dream in the legacy world.

So, now we have this jPOS world where we can take advantage of SQL databases.  But, as a jPOS implementer, you have some important responsibilities that go along with that shift in deployment models.   Here are the practices that you absolutely must get established:

• Backup nightly:  All defined tables ought to be backed-up as part of an overnight operation.
• Replicate real-time:  There are a sub-set of tables where a nightly backup isn’t sufficient enough protection from the results of an unexpected loss of data.  These tables are dynamic – they can be expected to change constantly throughout the day.  These tables need to be replicated on a real-time basis to a copy located on a separate server.  Ideally, all user and system tables in the primary database should be replicated to a failover database machine.
• Cull older days:  In older ‘legacy’ systems, a new file was created to house transactions associated with each specific capture date.  While this made file maintenance a nightmare, it had the positive effect of limiting the size of any file (and its associated indices) to a single day’s worth of transactions, meaning that performance of the transaction engine would remain relatively constant over time (assuming all other factors remained unchanged).  In a jPOS-based approach, the authorization engine makes use of SQL technology to facilitate its underlying DB requirements.  The advantages of employing SQL technology are dramatic: development, test, support, reporting and offline integration efforts are probably 2 to 3x easier (and surely far less esoteric) with SQL underpinnings vs. the proprietary approaches of the past. 
  However, using SQL technology does raise the bar in terms of proactively ensuring that the application does not inexorably lose its performance edge over time.  Namely, the transaction log houses all activity from all capture dates.  This log must not be allowed to hold an ever-increasing number of days of activity.  Otherwise, the high standards of performance demonstrated during user acceptance testing will slowly ebb away.  It stands to reason that an online system that has to fight through indices reflecting (e.g.) one year’s worth of production traffic will be hard pressed to perform at an acceptable speed.  The number of days to keep online (i.e., within the confines of the TranLog) will vary according to processing power and transaction volume.  Older data ought to be moved to a separate table location, where it can be accessed for historical query purposes and not negatively encumber online processing speed. 
  In conclusion, the size of the tranLog needs to be proactively managed each evening through automated, scheduled routines that cull each day as it falls of the end of the online range.

• Defragment Indices:  From experience, we know that regular execution of index defragmentation will result in faster performance, compact indices and a re-establishment of ‘target’ fill factors. 

In part 1, I reviewed “the critical skills that team members should have in order to succeed at a jPOS-based project.” To recap briefly, Alejandro said those were (in loose order of importance): O-O Programming; Java; Open Systems Projects; Hibernate; and jPOS itself.  [I added general SQL knowledge to the list.]

Now, the fact of the matter is, in a real-life project not every team member is going to meet the hurdle of this ‘ideal’ profile.  But you can still construct a very effective team.  Here are some thoughts on a good team based on some recent experiences.

1. There’s no getting around the fact that you need at least one lead developer that possesses all those talents (from my original list).  In fact, I’d go one step further and state that this person needs to have some additional skills in the kitbag: first, they ought to know about Online Transaction Processing (‘OLTP’ – inherently different from and more challenging than batch coding) and (optimally) they should have previous experience with payment systems (a.k.a., ‘financial switches’).  The litmus test on that last item is: is this person familiar with the ISO8583 standard?  If so, that’s your leader.  This person needs to do (or, head up) the implementation of the project’s core OLTP aspects.
2. Now, you might have a second person (or group) that comes close talent-wise, but is a bit junior to the person described in item 1 above.  Perhaps they possess many of the original skill list items, but are less experienced in OLTP or are brand-new to the payment systems world.  These people can be very effective by helping on the batch aspects of the project.  Don’t underestimate your offline needs.  Yeah, it’s sexy and cool to work on the OLTP stuff, but you’re not going to rollout until you’ve successfully built extract (settlement) files, created reports for key user constituencies, and built access/status screens for operations.  Don’t leave these tasks until the end of the project.  Get this part of the team working in conjunction with the team members from point #1.  In our projects, we make OLTP implementation decisions every day that have some type of impact on batch processing.  In an ideal world, you’ve got these two efforts going side-by-side.  The side benefit is that Team #2 learns a bit more about OLTP and Payment Systems every day the joint efforts go on.
3. Unlike legacy payment systems, jPOS is SQL-based.  As you saw in my previous post, I feel very strongly that having generic SQL knowledge is invaluable in these projects (I tacked the skill onto the end of Alejandro’s critical skill list).   I mention ‘generic’ here.  What I mean by that adjective is that this person doesn’t need to be an administrator.  [You must have access to a DBA to do the project and maintain performance levels, but not as a full-time project member.  See more thoughts on DBAs here.  The DBA is your best friend on a jPOS project.]  Instead, what I mean is that you need a full-time project team member who is comfortable devising, implementing and performing SQL-based tasks (in the DB of choice, e.g., MS SQL Server, MySQL etc.).  This person can be invaluable resource.  They can set up specific test conditions, allow iterative testing of new or changed functionality, assist in priming and maintaining production databases, and – most importantly (in my book) – allow Teams 1 and 2 (from above) to focus on the coding implementation.  That defines a nice ‘point of intersection’ where the coders produce the incremental schema additions/changes (out of Hibernate) and it’s up to Person #3 here to implement the schema and populate it.
4. [Now, we go out a bit from the core technical circle of the project...]  Next, you need someone to write specifications.  This is a critically important role.  [And this isn't simply because I write the specs on our projects.]  Here’s why: when you get these specs from organizations like FDR, AMEX, Discover (etc.), you quickly see that they need to be all things to all people.  They’ve got every possible type of option and implementation ‘mode’ defined.  Most times, maybe 20% (or less) of that content will apply to you.  It’s not fair to leave it to the coders to weed through the other 80% to figure out what’s ‘in play.’  My goal in these jPOS projects is that I always that I produce and provide a spec that eliminates any need for the technical team members to refer to the original source spec.  If they’re having to look through doc from AMEX to figure out an unclear point regarding the implementation, then my spec isn’t good enough.   These specs should synthesize all aspects of the project and leave nothing to guessing.
5. You also want to have someone in charge of communicating to third-parties.  Especially on these big projects where you’re building interfaces to three or more external organizations (not uncommon if you’re putting in Debit/EBT, Credit and one or more Stored Value links), the communication alone can roll down the hill like a growing snowball.  It’s important that a team member be designated to handle that communication and protect the others from ad-hoc and random communication attempts. 
6. You also need one or more testers.  I’m referring here to testers that are engaged before you have the users take a whack at it.  Typically, projects like this run pretty tight on manpower basis, so you often don’t have the luxury of a separate test team.  I know we typically don’t, so the spec writers (the people who really know the expected results the best) and the Third-Party communication person (from my points #5 and #6 above) may need to pull double-duty here.
7. One last point is about telecommunications, which typically manifests itself when you’re setting up connections to the third-parties (e.g., an online interface to First Data).  Now, the good part of this point is that with the standardization of everything onto TCP/IP (thank god), a full-time comm expert on your team isn’t a prerequisite.  [NOTE: if someone suggests anything but TCP/IP for connectivity, push back like a madman.  I'm serious.]  Of course, there’s a bad part, too: telecommunications has become increasingly complex due to security concerns and the overwhelming digitization of (and connecting to) everything.  So, like the DBA, you need to have someone (or a group) in the enterprise that you can call at a moment’s notice to ask the inevitable question: Why can’t we connect?

Readers of this blog know that I spend a lot of time talking about the intricacies of the various payment processing (acquirer-side) reversal scenarios and the importance of getting the reversal model right. 

Here’s another take at reversal processing – entirely TMI (Too Much Information) as the title of this piece suggests, plus some other thoughts on a ‘close cousin’ of reversal processing: finding and quashing duplicate transactions that could be tossed your way from the transaction’s origination point.

jPOS implementations can efficiently support the reversal and ‘find duplicate’ models of our clients. 
On the reversal side of an implementation, Acquirers need to support both host- and terminal-based reversals.  The host-based reversal side follows a very standard approach:

1. A timeout parameter is applied to all remote queries.
2. If no response is received prior to the expiration of the timeout, a reversal is generated (only if required – credit interfaces rely on settlement files to post and typically do not want to see reversals…even of the ‘courtesy’ variety).
3. The transaction is logged to the jPOS application’s ‘tranlog’ with a non-zero internal result code (‘irc’) and a ‘reversal indicator’ of ‘H’ reflecting a host-based reversal.

To show how this is implemented within a jPOS Transaction Manager, we provide here an example of two participants used to perform a ‘query host or reverse’ operation from an Acquirer to an FDR Debit/EBT financial gateway:

  <participant class=”org.jpos.ev.QueryHost”
     logger=”Q2″ realm=”query-remote-host”>
   <property name=”mux”     value=”fdr-mux” />
   <property name=”saf”     value=”saf” />
   <property name=”timeout” value=”25000″   />
   <property name=”threshold” value=”12000″   />
   <property name=”checkpoint” value=”query-host-or-reverse” />
   <property name=”reverse-on-timeout” value=”true” />
  </participant>  <participant class=”org.jpos.ev.FlagReversal”>
   <property name=”reversal-class” value=”H” />
  </participant> 

Terminal-based reversal processing is more of a challenge because the jPOS processing engine must be aligned to match up to the proprietary reversal model used by our customers.  We urge our clients to get their terminal-based reversal model details on the table Day 1 so we can get all the nuances worked out right away.  This means getting ‘origination point’ systems personnel to explain how they expect a reversal to be tied an original and then propagating that model into your jPOS implementation. 

We make special note of this point in order to pass along a tough lesson learned recently…our standard ‘point of acquisition’ timeout reversal model implementation assumed that the Primary Account Number (‘PAN’) was the linchpin around which we should base our reversal logic.  That turned out to be a faulty assumption; the store systems folks explained their front-end implementation and described a rather logical sequence of events where one card could be substituted for another (within the same customer session), the result being that a reversal may in fact have a different card number.

The important thing is that this isn’t a discussion on whether that front-end model is right, wrong, flawed, etc.  The reality is:  it’s reflective of a specific business model; it’s typically been proven right and improved/hardened over time; and – most importantly – the goal of any financial switch implementation should be to get into production without inflicting any type of change on a store systems implementation.  Since store systems changes most often manifest themselves in chain-wide rollouts, they should be avoided at all costs.

The terminal-based reversal is implemented via a ‘FindOriginal’ participant.  For example, here is the group of participants called at one jPOS client location for a terminal-based Debit Reversal transaction:

<group name=”DebitSaleReversal”>
  <participant class=”org.jpos.ev.PopulateDebitTranLog”
     logger=”Q2″ realm=”populate-debit-tranlog”>
   <property name=”itc” value=”05301″ />
   <property name=”cardType” value=”DB” />
   <property name=”checkpoint” value=”populate-debit-tranlog” />
  </participant>

  &validate_terminal;

  <participant class=”org.jpos.ev.FindOriginal”>
   <property name=”original-itc” value=”05300″ />
   <property name=”original-card-type” value=”DB” />
   <property name=”reversal-window” value=”86400″ />
  </participant>
  <participant class=”org.jpos.ev.CreateFDRRequest”
     logger=”Q2″ realm=”create-fdr-request”>
   <property name=”mti”        value=”0400″ />
   <property name=”pcode”      value=”009000″ />
   <property name=”template”   value=”cfg/fdr-template.xml” />
   <property name=”space”      value=”jdbm:fdr-stan” />
   <property name=”checkpoint” value=”create-fdr-request” />
  </participant>

  &store_and_forward;

  <participant class=”org.jpos.ev.FlagReversal”>
   <property name=”reversal-class” value=”T” />
  </participant>
  &force_approval;
  &debit_response;
</group>

In that participant flow (noting highlights here only)…

• The Debit Reversal (identified in a ‘switch’ participant prior to the flow shown here) is assigned an internal tran code (‘itc’) of 05301.  [NOTE:  These itc values aren't a jPOS standard; it's a Transaction Code numbering scheme I invented and have propagated throughout various jPOS-based solution implementations.]
• FindOriginal searches the tranlog for a corresponding original (making use of tranlog indices put in place by DBAs in support of this specific, important action).
• If an original was located, a reversal (0400 in this implementation) is formatted and placed into the appropriate store and forward (‘SAF’) queue.
• If an original was located, the FlagReversal participant tags the original with a ‘T’ (to signify a terminal-based reversal) in the Reversal Indicator field and cross-links the original and reversal in the tranlog.
• The ‘force_approval’ step ensures that if a corrupt, non-processable reversal is received from the origination point, we respond with an Approval in order to prevent a hard loop between terminal and host.
• The ‘debit_response’ step formats a message back to the origination point.  We take special care to respond to all reversal attempts – regardless of the internal result – with an Approval.  To do anything else is to invite the possibility of endless loops and, ultimately, manual intervention and queue clearance at the origination point.

In order to be reversed by FindOriginal:

• There needs to be a match (between an original and a reversal) on the fields designated by the implementer as the “reversal match-up fields.”  For example, at one jPOS/OLS.Switch client location, this match-up logic is implemented via a Hibernate call like this:

Criteria crit = db.session().createCriteria (DebitTranLog.class).
    add (Expression.eq (“storeNumber”, rev.getStoreNumber())).
    add (Expression.eq (“registerNumber”, rev.getRegisterNumber())).
    add (Expression.eq (“registerTranId”, rev.getRegisterTranId())).
    add (Expression.eq (“tenderNumber”, rev.getTenderNumber())).
    add (Expression.eq (“internalTranCode”, originalTranCode)).
    add (Expression.eq (“internalResultCode”, TRAN_APPROVED)).
    add (Expression.eq (“reconId”, new Long (0L))).
    add (Expression.isNull (“revInd”)).
    addOrder (Order.desc (“id”)).
    setMaxResults (1);

• The original transaction must have been approved.
• The original transaction must not have already been reversed.
• The original transaction must not have already been extracted in the nightly reconciliation process.

For further reading on reversal challenges and our team’s real-world experiences in this area, please refer to the following in-depth discussions…

Handling Acquirer-side Reversals in a Payment Switch

Get the Reversal Model right

Linking Originals and Reversals

The action of filtering out duplicates can be seen as a very ‘close cousin’ of the terminal-based reversal.  In both cases, the host implementation bears the responsibility of doing a “FindOriginal.”  In reversal processing, it is a reversal that prompts a look for an original (i.e., a financial request like a Purchase/Sale or Merchandise Return).  By contrast, in a ‘dup-check’ implementation every financial request will perform a FindOriginal-like step. 

For example, in the Debit Sale (Purchase), we execute (at one client site) this flow of participants:

<group name=”DebitSale”>
  <participant class=”org.jpos.ev.PopulateDebitTranLog”
     logger=”Q2″ realm=”populate-debit-tranlog”>
   <property name=”itc” value=”05300″ />
   <property name=”cardType” value=”DB” />
   <property name=”checkpoint” value=”populate-debit-tranlog” />
  </participant>

  &validate_terminal;

  <participant class=”org.jpos.ev.FindDuplicate”>
   <property name=”itc” value=”05300″ />
   <property name=”checkpoint” value=”find-duplicate” />
  </participant>
  <participant class=”org.jpos.ev.HasEntry”>
   <property name=”name” value=”DUPLICATE_TRANLOG” />
   <property name=”yes”  value=”DuplicateDebitResponse LogAndReply” />
   <property name=”no”   value=”DebitSale_Response LogAndReply” />
  </participant>
</group>

<group name=”DebitSale_Response”>
  &translate_pin;

  <participant class=”org.jpos.ev.CreateFDRRequest”
     logger=”Q2″ realm=”create-fdr-request”>
   <property name=”mti”        value=”0200″ />
   <property name=”pcode”      value=”009000″ />
   <property name=”template”   value=”cfg/fdr-template.xml” />
   <property name=”space”      value=”jdbm:fdr-stan” />
   <property name=”checkpoint” value=”create-fdr-request” />
  </participant>
  &query_fdr_host_or_reverse;
  &debit_response;
</group>

Behind the scenes, that ‘FindDuplicate’ participant is modeled very closely on ‘FindOriginal.’  The logic is very easy to follow here:

If ‘FindDuplicate’ locates a previous attempt (one that was approved and not subsequently reversed), it places a “DUPLICATE TRANLOG” entry into the transaction context.  The ‘HasEntry’ participant re-directs the transaction flow based upon whether the transaction in flight is determined to be a duplicate.  If it is not, the transactions proceeds as intended (here, the PIN is translated and external authorization is sought).  If it is, the transaction is given an internal result code (‘irc’) to reflect it as a Duplicate.  Of course, the origination point is told this transaction is an approval (the internal logic provides the approval code from the original).

For the record, the DuplicateDebitResponse participant is the referenced example looks like this: 

<group name=”DuplicateDebitResponse”>
  <participant class=”org.jpos.ev.SetRC” logger=”Q2″ realm=”duplicate-debit-response-set-rc”>
   <property name=”rc” value=”4001″ />
  </participant>
  &debit_response;
</group>