Microsoft – Windows \ SQL Server on HP Servers using Brocade\Mezzanine HBA Card – Error Source – bfad – Error Message – Heartbeat of IOC has failed

Microsoft – Windows \ SQL Server on HP Servers

When I first move to the Bay Area, I had the good fortune of working with one of the giant publishing companies.

I remember we had a Box that kept crashing.  Finally, the vendor owned up to it and sent us a replacement box.

What brings it to mind is that the in-house OS Administrator was literally crying that finally the vendor believed him.  And, shall we say he was able to sleep as the old server was a lemon.

Read more about California’s (CA) Motor Vehicle Warranty & Lemon Law
http://oag.ca.gov/consumers/general/lemon

These days you can sleep better.  You have the Internet and a community of techies writing down their experiences.

And, thank goodness for Google.  Just know what to look for and since it is so fast you have many gos at it.

Each is own, I do not read anymore – I just google it.  No wonder our bookstores are out of business; less to talk of community\city libraries.

Back to our discussions..

So on your Windows Box, if you start seeing errors such as:

1) In your Windows Event Viewer Log

– Branch : System
– Source : bfad
– Level : critical
– Message : Heartbeat of IOC has failed

2) And, on your Brocade\Mezzanine “Host Connectivity Manager” \ HCM Log will contain something close to this:

3) And, today is not the day you will go completely crazy nor start sobbing at work

Then look to your Brocade Mezzanie card that is quite common with HP Blade Servers.

HP Description states:

This occurs because the Brocade 804 mezzanine card BIOS incorrectly self-configures architecture internal to the ASIC, resulting in data throughput that can cause brief losses of the fabric connection. Occurrences may be more noticeable with an increased number of LUNs.

The error was fixed in BIOS revision 3.0.3.4. And, the post is dated June 5th, 2012.

So five months later it bit us.

To remediate:

  • Apply “HP Firmware Online Flash for Brocade Fibre Channel Host Bus Adapters” version 2012.05.01 or later

HP’s excellent and most useful write-up is available @

Advisory: Brocade 804 8Gb Fibre Channel HBA for HP BladeSystem c-Class- Adapter May Experience a Brief Loss of Fabric Connection on All Ports

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c03362107&lang=en&cc=us&taskId=135&prodSeriesId=3676868&prodTypeId=15351

As of election day 2012 the latest patch is

Type: Firmware – Storage Fibre Channel
Version: 2012.10.01 (26 Oct 2012)
File name: cp018011.exe (2.5 MB)

And, it is available @

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&swLang=8&mode=2&taskId=135&swItem=MTX-98c4a6eb74a64a2c9e40489cff

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s