Avoiding the IDA Pro-over-RDP hang issue

Note: This is fixed in IDA since version 6.7

If you are an enthusiastic IDA Pro user like myself and use more than one Windows computer, chances are you've hit an issue when using IDA Pro in a RDP session. Specifically, attempting to dock/undock IDA windows results in a hang of the RDP session. This issue was reported but is unlikely to be fixed, so I have just been patching IDA to prevent the issue from occurring while maintaining functionality. Recently Hex-Rays have made numerous releases in a small time frame, making repeated patching required, which is highly annoying.

The following is an IDAPython script to patch IDA:

#ida-rdp-patch.py
import shutil  
import os  
import struct  
from idaapi import *  
from idc import *  
from idautils import *

grabWidget_ref = get_first_cref_to(get_name_ea(0, '?grabWidget@QPixmap@QT@@SA?AV12@PAVQWidget@2@HHHH@Z'))  
setPixmap_ref = get_first_cref_to(get_name_ea(0, '?setPixmap@QDrag@QT@@QAEXABVQPixmap@2@@Z'))

grabWidget_callSize = DecodeInstruction(grabWidget_ref).size  
jmp_ea = grabWidget_ref + grabWidget_callSize + DecodeInstruction(grabWidget_ref + grabWidget_callSize).size  
jmp_dst = setPixmap_ref + DecodeInstruction(setPixmap_ref).size  
jmp_filepos = get_fileregion_offset(jmp_ea)  
jmp = struct.pack('BB', 0xeb, jmp_dst - jmp_ea - 2)

print 'insert jmp @ %8x (filepos %8x) to %8x' % (jmp_ea, jmp_filepos, jmp_dst)

inputPath = GetInputFilePath()  
inputFile = os.path.splitext(GetInputFile())  
outputPath = os.path.expanduser('~') + '/Desktop/' + inputFile[0] + '-rdp' + inputFile[1]  
print '%s -> %s' % (inputPath, outputPath)  
shutil.copy2(inputPath, outputPath)

with open(outputPath, 'r+b') as fo:  
    fo.seek(jmp_filepos)
    fo.write(jmp)

This post was originally published on Thursday 6 February 2014, 09:56