Xilinx tools on Windows 8

Note: Xilinx fixed this at some point, for new tools.
The instructions may still be helpful for older tools/versions.

Recently I wanted to use Xilinx tools like iMPACT, Project Navigator, and Vivado on Windows 8 x64. As it turns out, there are issues with running on Windows 8 in general, and also with Windows x64 compatibility. A quick search led me to helpful posts. While that is a good starting point, I wanted everything to work.
The following details the existing issues and how to fix them such that all Xilinx programs (32 and 64 bit) run correctly on Windows 8/8.1:

Remove Windows 8 Restriction (32bit and x64)

For some reason, Vivado is hard-coded to fail on Windows 8. The function librdi_common!HUTEnv::isWindows8 looks like:

bool __cdecl HUTEnv::isWindows8()  
{
  OSVERSIONINFOA v1; // [sp+0h] [bp-98h]@2

  if ( !g_windows_version )
  {
    memset(&v1, 0, 0x94u);
    v1.dwOSVersionInfoSize = 0x94;
    GetVersionExA(&v1);
    if ( v1.dwPlatformId == 2 && v1.dwMajorVersion == 6 )
    {
      if ( v1.dwMinorVersion == 1 )
      {
        g_windows_version = 7;
      }
      else if ( v1.dwMinorVersion == 2 )
      {
        g_windows_version = 8;
      }
    }
  }
  return g_windows_version == 8;
}

If this returns true, then the caller will throw a fatal C++ exception. Just patch the function to return 0.

Disable SmartHeap (x64 only)

shsmp (SmartHeap for SMP)

Xilinx has a shared library SHSMP{64}.dll which patches many CRT functions. This causes multiple problems on Windows 8 and x64 environments, such as an infinite loop while applying patches (causing stack overflow), and incorrect patches (causing crashes from read/write access violations during runtime). The DllMain function of this library only calls shi_patchMallocs to apply the patches. Just patch the dll entry point to return 1 and therefore bypass the shi_patchMallocs call.

libPortability

Xilinx ships a library - libPortability.dll - which directly uses SmartHeap. Since we disable it in a rather hacky way, we need to prevent it from being used at all. Luckily Xilinx has provided a second version of libPortability which doesn't use the shim engine - libPortabilityNOSH.dll. Personally I like to rename the existing library and symlink to the proper version:

move libPortability.dll libPortabilitySH.dll  
mklink libPortability.dll libPortabilityNOSH.dll  

Apply Globally

I installed the complete ISE and Vivado packs, which results in many copies of the libraries we need to patch being strewn around. Make sure you find all of them and replace/symlink with the patched version.
For example, shsmp64.dll was found in all of the following directories:

• 14.6\ISE_DS\.xinstall\bin\nt64
• 14.6\ISE_DS\common\lib\nt64
• 14.6\ISE_DS\EDK\lib\nt64
• 14.6\ISE_DS\ISE\lib\nt64
• 14.6\ISE_DS\PlanAhead\lib\win64.o
• DocNav\.xinstall\bin\nt64
• SDK\2013.2\lib\nt64
• Vivado\2013.2\.xinstall\bin\nt64
• Vivado\2013.2\ids_lite\ISE\lib\nt64
• Vivado\2013.2\lib\win64.o

libPortability also exists in many places, just run a dir /s to find it :)

Make some new shortcuts

At least for me, there were no shortcuts created by the installer for 64bit iMPACT and 32bit Vivado. Creating them is straightforward:
(%XILINX_DIR% is e.g. C:\Xilinx\)

iMPACT (and other ISE programs)

target: %XILINX_DIR%14.6\ISE_DS\settings64.bat %XILINX_DIR%14.6\ISE_DS\ISE\bin\nt64\impact.exe  
working dir: %XILINX_DIR%14.6\ISE_DS\ISE  

For 32bit, use settings32.bat and bin\nt\ directory.

Vivado

target: %XILINX_DIR%Vivado\2013.2\bin\unwrapped\win64.o\vvgl.exe %XILINX_DIR%Vivado\2013.2\.\bin\vivado.bat  
working dir: %appdata%\Xilinx\Vivado  

For 32bit, use bin\unwrapped\win32.o\ directory.

Conclusion

I can finally start playing with the Zedboard I got :)

This post was originally published on Monday 21 October 2013, 12:25