Firmware Update Failure on XL710

Wi-Fi & Networking Ethernet Products
1

We are trying to use SR-IOV on the Intel XL710 cards (2-port, 40G) and have issue with initialization.

According to 29. I40E Poll Mode Driver — Data Plane Development Kit 26.07.0 documentation, the matching FW version  for the DPDK version (17.02) we are using should be 5.05, and firmware on my card is 6.03.

So I was trying to downgrade the XL710 firmware from 6.03 to 5.05 using the Linux version of nvmupdate64e, but I forgot to shutdown the VM which is using the card, and the update failed.

Trying to recover, I shut down the machine completely, and the power the system back up. Now I will get the following error message during the boot:

======

[ 2.736693] i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 1.6.27-k
[ 2.736695] i40e: Copyright (c) 2013 - 2014 Intel Corporation.
[ 6.720801] i40e 0000:04:00.0: fw 6.0.48442 api 1.7 nvm 5.05 0x800028a6 1.1747.0
[ 6.720803] i40e 0000:04:00.0: The driver for the device detected a newer version of the NVM image than expected. Please install the most recent version of the network driver.
[ 6.952890] i40e 0000:04:00.0: capability discovery failed, err OK aq_err I40E_AQ_RC_EMODE
[ 10.961130] i40e 0000:04:00.1: fw 6.0.48442 api 1.7 nvm 5.05 0x800028a6 1.1747.0
[ 10.961137] i40e 0000:04:00.1: The driver for the device detected a newer version of the NVM image than expected. Please install the most recent version of the network driver.
[ 11.191554] i40e 0000:04:00.1: capability discovery failed, err OK aq_err I40E_AQ_RC_EMODE

=====

What should I do to recover the NIC? Your help are greatly appreciated.

2

Your link is bad.

Doc (not an Intel employee or contractor)

3

Here is the link again:

https://doc.dpdk.org/guides/nics/i40e.html

4

Hi Jing,

We would like to check if the suggestions of our esteemed community members addressed your concern.

We look forward to hearing from you. Should we not hear from you, we will follow up after 3 business days.

Best Regards,

Alfred S

Intel® Customer Support

5

No, he has not addressed my concern.

6

Hi Jing,

Thank you for your response.

We are sorry to know that you are still experiencing the issue.

Please allow us to ask more questions; in this way, we can determine what would be the best recommendation for your concern:

1. May we know the exact model of the Intel XL710?

2. We need to know if your card is an OEM variant or our very own retail variant. OEM variants may have different firmware than the retail versions. To determine this, if possible, kindly give us a picture focusing on the markings printed on the different stickers on the card. There will also be marking embossed or printed on the card’s circuit board, please also provide a picture of it.

We look forward to your reply. Should we not get your reply, we will follow up after three business days.

Best Regards,

Alfred S

Intel® Customer Support

7

Alfred,

Thanks for your help.

We have a XL710-QDA2 card.

Here are a couple of pictures for the 40G NIC we are using.

Sam

PXL_20201217_161851141.jpg
PXL_20201217_161851141.jpg3301×1645 1.07 MB

PXL_20201217_161938777.jpg
PXL_20201217_161938777.jpg3825×2496 1.95 MB

8

Hi Sam65021,

Thank you for your response.

Kindly provide us the results of this command: ethtool -i ethx where ethx is the Ethernet port.

We look forward to your reply. Should we not get your reply, we will follow up after three business days.

Best Regards,

Alfred S

Intel® Customer Support

9

Alfred,

The 40G Ethernet card is not recognized, so I can not run ethtool command on it.

The dmesg output is list below.

Sam

======

[    2.738019] i40e: Intel(R) Ethernet Connection XL710 Network Driver - version 1.6.27-k

[    2.738021] i40e: Copyright (c) 2013 - 2014 Intel Corporation.

[    6.701470] i40e 0000:04:00.0: fw 6.0.48442 api 1.7 nvm 5.05 0x800028a6 1.1747.0

[    6.701473] i40e 0000:04:00.0: The driver for the device detected a newer version of the NVM image than expected. Please install the most recent version of the network driver.

[    6.933172] i40e 0000:04:00.0: capability discovery failed, err OK aq_err I40E_AQ_RC_EMODE

[   10.942043] i40e 0000:04:00.1: fw 6.0.48442 api 1.7 nvm 5.05 0x800028a6 1.1747.0

[   10.942046] i40e 0000:04:00.1: The driver for the device detected a newer version of the NVM image than expected. Please install the most recent version of the network driver.

[   11.172354] i40e 0000:04:00.1: capability discovery failed, err OK aq_err I40E_AQ_RC_EMODE

10

Hi Sam65021,

Thank you for your response.

We are already starting for look closely on your issue. We have questions for your to aid us in find a recommendation for your concern. Please answer the following questions:

1. May we know the exact Linux based OS that you are using?

You may run our Intel® System Support Utility from this page, https://downloadcenter.intel.com/download/26735/Intel-System-Support-Utility-for-the-Linux-Operating-System#:~:text=Intel%20SSU%20for%20the%20Linux,and%20shared%20by%20the%20user. After running it, you will be given an option to save the logs to a text file, please do so and attach the file on your reply, in this way, we will be the ones to check the OS version.

2. To confirm are you already using the most updated network driver for your Linux OS?

3. If you have already updated it, please try to uninstall the current driver and perform a clean install of the driver on your system and see if the same error occurs. If the same error happens, please try to use an older driver version and see if that bypasses the error message.

We look forward to your reply. Should we not get your reply, we will follow up after three business days.

Best Regards,

Alfred S

Intel® Customer Support

11

Alfred,

1. We are using Redhat Enterprise Linux 7.4

john@redhat-linux ~]$ cat /etc/redhat-release

Red Hat Enterprise Linux Server release 7.4 (Maipo)

2. The driver version is 1.6.27

[john@redhat-linux ~]$ modinfo i40e

filename:       /lib/modules/3.10.0-693.el7.x86_64/kernel/drivers/net/ethernet/intel/i40e/i40e.ko.xzversion:        1.6.27-k

3. Note that the card works fine until I tried to downgrade the firmware using  nvmupdate64e.

4. If I run nvmupdate64e, I will get the following error message:

[john@redhat-linux Linux_x64]$ sudo ./nvmupdate64e

Intel(R) Ethernet NVM Update Tool

NVMUpdate version 1.30.22.3

Copyright (C) 2013 - 2017 Intel Corporation.

WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.

Inventory in progress. Please wait [*…]

Num Description                               Ver. DevId S:B    Status

=== ======================================== ===== ===== ====== ===============

01) Intel(R) I350 Gigabit Network Connection  1.72  1521 00:002 Update not

available

02) Intel(R) Ethernet Converged Network             1583 00:004 Access error

Adapter XL710-Q2

Tool execution completed with the following status: Device not found

Press any key to exit.

12

Hi Sam65021,

Thank you for your response and providing more details regarding the issue,

We will continue checking this and the information that provided is invaluable to us.

Please allow us some time to check on this. We hope for your understanding regarding this

We will get back to you no later than 3 business days from now.

Best Regards,

Alfred S

Intel® Customer Support

13

Hi Sam65021,

Thank you for your patience in waiting for an update.

Due to the complexity of the issue, please allow us more time to check on your concern.

We will try to get back to you with an answer no later than 3 business days from now; although, we will reach out to you in case we need more information or if we already have developments regarding your concern.

Best Regards,

Alfred S

Intel® Customer Support

14

Hi sam65021,

Thank you for waiting for our update.

While we were closely checking your concern, we find the need to get a more extensive look at your network adapter due to the reason that firmware version 6.03 is not available on our download center for the card model that you are reporting, as we only have version 6.01 then after that 6.80. You may refer to this page for confirmation, https://downloadcenter.intel.com/download/24769/Non-Volatile-Memory-NVM-Update-Utility-for-Intel-Ethernet-Network-Adapter-700-Series.

For this reason, we would like to make sure about the model of your card.

Hence, we need to get the PBA number from your card. We need to determine if it is a retail adapter or if you have a different version. Having a different version means that it also needs a distinct firmware package compared to the one found on our download center.

Consequently, please provide us with pictures of the markings printed on the stickers on the card. Kindly provide us with pictures focusing on the characters printed on the sticker and the PBA number printed on the circuit board of the card.

We look forward to your reply. Should we not get your reply, we will follow up after three business days.

Best Regards,

Alfred S

Intel® Customer Support

15

Hi sam65021,

Thank you for waiting for our update.

While we were closely checking your concern, we find the need to get a more extensive look at your network adapter due to the reason that firmware version 6.03 is not available on our download center for the card model that you are reporting, as we only have version 6.01 then after that 6.80. You may refer to this page for confirmation, https://downloadcenter.intel.com/download/24769/Non-Volatile-Memory-NVM-Update-Utility-for-Intel-Ethernet-Network-Adapter-700-Series.

For this reason, we would like to make sure about the model of your card.

Hence, we need to get the PBA number from your card. We need to determine if it is a retail adapter or if you have a different version. Having a different version means that it also needs a distinct firmware package compared to the one found on our download center.

Consequently, please provide us with pictures of the markings printed on the stickers on the card. Kindly provide us with pictures focusing on the characters printed on the sticker and the PBA number printed on the circuit board of the card.

We look forward to your reply. Should we not get your reply, we will follow up after three business days.

Best Regards,

Alfred S

Intel® Customer Support

16

Hi Sam65021,

We are just following up.

It looks like you need more time to provide the information that we asked.

We will follow up again after 3 business days. Should we not hear from you, our system may automatically close the thread.

Best Regards,

Alfred S

Intel Customer Support

17

Hi Sam65021,

We need to close this thread since we have not gotten a response from you: maybe because you are busy or preoccupied at the moment. We know that this is important for you to get it resolved and it is also equally important for us to give you the right solution; as much as we would like to assist you, we need to close it to attend to other customers. We hope for your consideration and understanding on this one.

If you need any additional information, please submit a new question as this thread will no longer being monitored.

Thank you for contacting Intel® and have a great week!

Best Regards,

Alfred S

Intel® Customer Support