Praveen Katiyar' Blog

Learning along the way . . . .

Fetching a byte buffer from a Win32 DLL

 

Define  a function in native dll (let say it “Win32Native.dll”)  as shown below.

extern "C" __declspec(dllexport) int FetchByteArray ( int nSize, byte** ppnArray )
{
    int result = 0;
    //    CoTaskMemAlloc must be used instead of the new operator because code on the managed side will call
    //    Marshal.FreeCoTaskMem to free this memory.
    byte* newArray = (byte*)CoTaskMemAlloc( sizeof(byte) * nSize );
    for ( int j = 0; j < nNewSize ; j++ )
    {
        newArray[j] = ( j+1 ) % 255 ;
        result += newArray[j];
    }

    // release the previous buffer, if any allocated.
    if ( *ppnArray != NULL ) CoTaskMemFree( *ppnArray );
    *ppnArray = newArray;
    return result;
}

Point of Interest
  • CoTaskMemAlloc is used to allocated the memory required.
  • CoTaskMemFree is used to free any previously allocated buffer, if null is passed then, CoTaskMemFree is not called.

If you want to use a heap that is shared between native and managed, it is more common to use the COM heap.

Writing the client code (the managed part)

one can simple create a console base application which can use this dll. let’s name it MarshallingTest.

see the code snippet below.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;

namespace MarshallingTest
{
    class Program
    {
        [DllImport("Win32Native.dll")]
        public static extern int FetchByteArray(int nSize, ref IntPtr arrInt);
    
        static void Main(string[] args)
        {
            int nSize = 10;
            IntPtr ptrArr = IntPtr.Zero;

            int nSum = FetchByteArray(nSize, ref ptrArr);
            byte[] arrByte = new byte[nSize];
            Marshal.Copy(ptrArr, arrByte, 0, nSize);

            Console.WriteLine("\nReturned Buffer\n");

            for (int i = 0; i < nSize; i++)
            {
                Console.Write ( "{0:D2}  ", arrByte[i] );
            }
           
            Console.Write("\nSum of Buffer : {0}\n", nSum );
            Marshal.FreeCoTaskMem(ptrArr);
        }
    }
}

Point of Interest
  • namespace System.Runtime.InteropServices;  defines the declarations necessary for Interop operations, like DllImport,
  • DllImport defines the DLL entry point.
  • Marshal.Copy function used to copy buffer from managed buffer to unmanaged buffer and vice versa.
  • Marshal.FreeCoTaskMem frees the memory allocated by native DLL.

compile and execute you will get following output.

image

July 31, 2013 Posted by | .NET, CodeProject, Interoperability, Marshaling, Win32 | , , , | Leave a comment

Interoperability Returning a string from Wn32 DLL

Define  a function in native dll (let say it “Win32Native.dll”)  as shown below.

extern "C" __declspec(dllexport)  char * GetStringFromDLL()
{
    const size_t alloc_size = 128;
    STRSAFE_LPSTR result=(STRSAFE_LPSTR)
CoTaskMemAlloc(alloc_size);
    STRSAFE_LPCSTR teststr = "This is return string From Native DLL";
    StringCchCopyA ( result, alloc_size, teststr );
    return result;
}

Point of Interest

  • STRSAFE_LPSTR is a typedef of char *
  • StringCchCopy is a replacement for strcpy (with safety).  The size, in characters, of the destination buffer is provided to the function to ensure that StringCchCopyb does not write past the end of this buffer. has two variants.

Writing the client code (the managed part)

one can simple create a console base application which can use this dll. let’s name it MarshallingTest.

see the code snippet below.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Text;

namespace MarshallingTest
{
    class Program
    {
       [DllImport("Win32Native.dll")]
public static extern String GetStringFromDLL();

        static void Main(string[] args)
        {
            Console.WriteLine("Line displayed below is returned from a Native DLL");
            string strData = GetStringFromDLL();
            Console.WriteLine ( "Returned value [" + strData +"]" );
        }
    }
}

Point of Interest

  • namespace System.Runtime.InteropServices;  defines the declarations necessary for Interop operations, like DllImport,
  • DllImport defines the DLL entry point.

compile and execute you will get following output.

image

 

July 30, 2013 Posted by | CodeProject, Interoperability, Win32 | , , | Leave a comment

Interoperability with Winodws API (1 of N)

In this article I will explain, how one can call windows API, from managed world. let’s define the signatures of windows API for managed world.

Calling a Win32 MessageBox function from Managed Word

//****************************

// MyProgram.cs

namespace MyManagedProgram   {  //1

class Program   { // 2
static void Main(string[] args)   { //3

Win32DllWrapper.MsgBox(0, "This Text has been sent from  Managed World",          "Win32 Messsage Box", 0);

} //3

}  //2

///defining the helper class

///define a helper class which will contain a functions for

public class Win32DllWrapper   {    //4
//  Standard Functions from Windows API.
[DllImport("User32.dll", EntryPoint = "MessageBox", CharSet = CharSet.Auto)]
public static extern int MsgBox(int hWnd, String text, String caption, uint type);

}  //4

}  // 1

compile and execute

July 26, 2013 Posted by | .NET, CodeProject, Interoperability, Marshaling, Win32 | , , | Leave a comment

Marshaling Introduction (1 of N)

What is Marshaling?

Marshaling (similar to serialization) is the process of transforming the memory representation of an object to a data format suitable for storage or transmission. It is typically used when data must be moved between different parts of a computer program or from one program to another.

in the .NET scenario, Marshaling is the process of creating a bridge between managed code and unmanaged code; it is the homer that carries messages from the managed to the unmanaged environment and reverse. It is one of the core services offered by the CLR (Common Language Runtime.) Because much of the types in unmanaged environment do not have counterparts in managed environment, you need to create conversion routines that convert the managed types into unmanaged and vice versa; and that is the marshaling process. As a refresher, we call .NET code “managed” because it is controlled (managed) by the CLR. Other code that is not controlled by the CLR is called

When we need Marshaling

You already know that there is no such compatibility between managed and unmanaged environments. In other words, .NET does not contain such the types HRESULT, DWORD, and HANDLE that exist in the realm of unmanaged code. Therefore, you need to find a .NET substitute or create your own if needed. That is what called marshaling.

An example is the unmanaged DWORD; it is an unsigned 32-bit integer, so we can marshal it in .NET as System.UInt32. Therefore, System.UInt32 is a substitute for the unmanaged DWORD.

Marshaling comes handy when you are working with unmanaged code, whether you are working with Windows API or COM components. It helps you interoperating (i.e. working) correctly with these environments by providing a way to share data between the two environments. 

image

Interop Marshaling

Interop marshaling governs how data is passed in method arguments and return values between managed and unmanaged memory during calls. Interop marshaling is a run-time activity performed by the common language runtime’s marshaling service.

Most data types have common representations in both managed and unmanaged memory. The interop marshaler handles these types for you. Other types can be ambiguous or not represented at all in managed memory.

An ambiguous type can have either multiple unmanaged representations that map to a single managed type, or missing type information, such as the size of an array. For ambiguous types, the marshaler provides a default representation and alternative representations where multiple representations exist. You can supply explicit instructions to the marshaler on how it is to marshal an ambiguous type.

Default Marshaling

  • Platform invoke, which enables managed code to call functions exported from an unmanaged library.
  • COM interop, which enables managed code to interact with COM objects through interfaces.

Marshaling data with platform Invoke (PInvoke)

To call functions exported from an unmanaged library, a .NET Framework application requires a function prototype in managed code that represents the unmanaged function. To create a prototype that enables platform invoke to marshal data correctly, you must do the following:

  • Apply the DLLImportAttribute attribute to the static function or method in managed code.
  • Substitute managed data types for unmanaged data types.

You can use the documentation supplied with an unmanaged function to construct an equivalent managed prototype by applying the attribute with its optional fields and substituting managed data types for unmanaged types.

Memory Management

The interop marshaler always attempts to free memory allocated by unmanaged code. This behavior complies with COM memory management rules, but differs from the rules that govern native C++.

Confusion can arise if you anticipate native C++ behavior (no memory freeing) when using platform invoke, which automatically frees memory for pointers. 

see the following example

BSTR MyMethod (BSTR b)

{

            return b;

}

However, if you define the method as a platform invoke prototype, replace each BSTR type with a String type, and call Method, the common language runtime attempts to free variable b twice in the above example. You can change the marshaling behavior by using IntPtr types rather than String types.

The runtime always uses the CoTaskMemFree method to free memory. If the memory you are working with was not allocated with the CoTaskMemAlloc method, you must use an IntPtr and free the memory manually using the appropriate method. Similarly, you can avoid automatic memory freeing in situations where memory should never be freed.

Directional attribute

Directional attributes are optional. You apply them to method parameters when you want to alter the default behavior of the marshaler. If you omit directional attributes from a method parameter, the marshaler determines the directional flow based on the type of the parameter (value or reference) and its modifiers, if any.

Some languages provide keywords that enable you to modify the directional flow of method parameters. The following table lists the direction-related keywords provided by Visual Basic .NET and C# and shows the equivalent IDL interface attribute.

Default marshaling of method arguments to unmanaged code

ByRef, ref, and out parameter modifiers cause method arguments to be marshaled by reference rather than by value. Method arguments passed by value are marshaled to unmanaged code as values on the stack; arguments passed by reference are marshaled as pointers on the stack. The figure below shows the default marshaling behavior of value types and reference types with parameter modifiers.

image

By default, reference types (classes, arrays, strings, and interfaces) passed by value are marshaled as In parameters for performance reasons. You do not see changes to these types unless you apply InAttribute and OutAttribute (or just OutAttribute) to the method parameter. The StringBuilder class, which is an exception to this rule, is marshaled as an In/Out parameter.

The interop marshaler guarantees the following behavior with regard to directional attributes:

  • The interop marshaler never generates a write operation to an In parameter passed from unmanaged code. Thus, unmanaged code can safely pass a pointer to a read-only page, or a pointer to concurrently accessed data.
  • When the copied object contains an allocated object, such as a BSTR, the marshaler always executes the correct sequence of allocations and destructions demanded by the In/Out settings.

Default Marshaling

To call functions exported from an unmanaged library, a .NET Framework application requires

Blittable Types

Most data types have a common representation in both managed and unmanaged memory and do not require special handling by the interop marshaler. These types are called blittable types because they do not require conversion when passed between managed and unmanaged code.

  • System.Byte
  • System.SByte
  • System.Int16
  • System.UInt16
  • System.Int32
  • System.UInt32
  • System.Int64
  • System.IntPtr
  • System.UIntPtr
  • One-dimensional arrays of blittable types, such as an array of integers
  • Formatted value types that contain only blittable types (and classes if they are marshaled as formatted types).

The following table lists non-blittable types from the System namespace. Delegates, which are data structures that refer to a static method or to a class instance, are also non-blittable.

  • System.Array
  • System.Boolean
  • System.Char
  • System.Object
  • System.Mdarray
  • System.String
  • System.Valuetype
  • System.SzArray

July 26, 2013 Posted by | .NET, CodeProject, SOA, WCF | , , | Leave a comment

Packing a array inside a VARIANT (VC++ 6.0) (Series 3 of N)

This is the third post about variant, in this post I I am going to explain, how an array of structures can be packed inside a VARIANT variable. suppose we have a structure as definition shown below.

typedef struct MyStructType
{
    int nID ;
    long lVal ;
    double dblVal ;
    TCHAR szBuffer[255];
} MyStructType ;

Packing  a array of structure inside a Variant

int  PackVariantWithStructArray(short nCnt, VARIANT * pVar )
{
    // TODO: Add your dispatch handler code here
    USES_CONVERSION ;

    //Initialize the VARIANT (Type is SAFEARRAY of BYTE)
    VariantInit(pVar);
    pVar->vt = VT_ARRAY | VT_UI1 ;

    int nBufferSize = nCnt * sizeof ( MyStructType );

    // Define a safe array of nCnt Item and Starting index as 0
    SAFEARRAYBOUND safeBounds = { nBufferSize, 0};
   
    //Create the Safe Array passing it the bounds
    SAFEARRAY* pSafeArray = SafeArrayCreate(VT_UI1, 1, &safeBounds);

    //Get a pointer to the array data, This actually increments the array’s lock count)
    MyStructType * structArray = NULL;
    SafeArrayAccessData(pSafeArray, (void**)&structArray);
    for ( int i = 0 ; i < nCnt ; i++ )
    {
        CString strTmp ;
        strTmp.Format ( _T("This is Item %2d"), i );
        structArray[i].dblVal = (i +1) * 101.0 ;
        structArray[i].lVal = (i +1) * 11 ;
        structArray[i].nID = (i +1) ;
        _tcscpy ( structArray[i].szBuffer, strTmp) ;
    }

    // We are done wth populating the array Decrement the lock

     SafeArrayUnaccessData(pSafeArray);

    //Assign our VARIANT out param with the array   
    pVar->parray = pSafeArray ;
    return nCnt ;
}

as it can be seen from the above code, here I have just created a byte buffer sufficient to accommodate the all structures,

it involves following steps.

  1. Allocating the desired temporary buffer space to hold the array of structures (that should be equivalent to (number of structure x size of structure), and filling that array with values one need to return.
  2. creating a safe array of the desired (VT_U1) type.
  3. copying the temporary buffer to the safe array of the variant.
  4. free the memory allocated for temporary buffer. (avoid memory leaks).

Fetching a array of structure from a variant.

MyStructType * UnPackVariantWithStructArray(short nCount, VARIANT var )
{
    MyStructType *pBuffer = (MyStructType *) calloc ( nCount, sizeof (MyStructType)) ; 

    USES_CONVERSION ;
    SAFEARRAY* pSafeArray  = var.parray ;
    //Get a pointer to the array data, This actually increments the array’s lock count)
    MyStructType *structArray = NULL ;
    SafeArrayAccessData ( pSafeArray, (void**)&structArray );
    for ( int i = 0 ; i < nCount ; i++ )
    {
        pBuffer[ i].dblVal = structArray [i].dblVal  ;
        pBuffer[ i].lVal = structArray [i].lVal ;
        pBuffer[ i].nID = structArray [i].nID   ;
        pBuffer[ i].dblVal = structArray [i].dblVal  ;
        _tcscpy ( pBuffer[ i].szBuffer, structArray [i].szBuffer );
    }
    //    We are done wth populating the array Decrement the array’s lock count
    SafeArrayUnaccessData(pSafeArray);
    return pBuffer ;
}

this is a crude method, although there is a better method using type libraries, that I will explain in some other article.

July 19, 2013 Posted by | CodeProject, Interoperability, Win32 | , , | 2 Comments

Packing a array inside a VARIANT (VC++ 6.0) (Series 2 of N)

This is the second post about variant, in this post I I am going to explain, how an array of strings can be packed inside a VARIANT variable.

Packing  a array of string inside a Variant

int  PackVariantWithStringArray(short nCnt, VARIANT * pVar )
{
    USES_CONVERSION ;

    //Initialize the VARIANT (Type is SAFEARRAY of BSTRs)
    VariantInit(pVar);
    pVar->vt = VT_ARRAY | VT_BSTR;

    // Define a safe array of nCnt Item and Starting index as 0
    SAFEARRAYBOUND safeBounds = { nCnt, 0};
   
    //Create the Safe Array passing it the bounds
    SAFEARRAY* pSafeArray = SafeArrayCreate(VT_BSTR, 1, &safeBounds);

    //Get a pointer, actually increments the array’s lock count)
    BSTR* bstrArray = NULL;
    SafeArrayAccessData(pSafeArray, (void**)&bstrArray);
    for ( int i = 0 ; i < nCnt ; i++ )
    {
        CString strTmp ;
        strTmp.Format ( _T("This is Item %2d"), i );
        bstrArray [i] = strTmp.AllocSysString () ;
    }

    //done wth populating, Decrement the array’s lock count
    SafeArrayUnaccessData(pSafeArray);

    //Assign our VARIANT out param with the array   
    pVar->parray = pSafeArray ;
    return nCnt ;
}

as it can be seen from the above code, it involves following steps.

  1. Allocating the desired temporary buffer space to hold the array of bytes, and filling that array with values one need to return.
  2. creating a safe array of the desired (VT_BSTR) type.
  3. copying the temporary buffer to the safe array of the variant.
  4. free the memory allocated for temporary buffer. (avoid memory leaks).

Fetching a array of strings from a variant.

TCHAR ** UnPackVariantWithStringArray(short nCnt, VARIANT var )
{
    TCHAR **pBuffer = (TCHAR **) calloc ( nCnt, sizeof (INT)) ; 

    USES_CONVERSION ;
    SAFEARRAY* pSafeArray  = var.parray ;
    //pointer to array data, actually increments the lock count
    BSTR* bstrArray = NULL ;
    SafeArrayAccessData ( pSafeArray, (void**)&bstrArray);
    for ( int i = 0 ; i < nCnt ; i++ )
    {
        LPTSTR szTmp = OLE2T ( bstrArray [i] ) ;
        int nLen = _tcslen ( szTmp );
        pBuffer [i]= (TCHAR *) calloc ( nLen, sizeof ( TCHAR));
        _tcscpy ( pBuffer [i], szTmp);
    }
   
    //Done wth populating, Decrement the array’s lock count
    SafeArrayUnaccessData(pSafeArray);
    return pBuffer ;
}

these conversion is useful in middleware scenario. in the next article I will explain how to pack array of structure inside a variant.

July 18, 2013 Posted by | CodeProject, MFC, Win32 | , , | Leave a comment

Packing a array inside a VARIANT (VC++ 6.0) (Series 1 of N)

VARIANTs are versatile data types, they can hold any type of value, apart from that they can hold array of different types. In this post I am going to explain how can you pack a array of bytes, inside a variant variable.

Packing  a array of simple types inside a Variant

Packing  a array of bytes

int GetVariantWithByteBuffer (int nBufferSize, VARIANT FAR* pVar)
{

    /* here size of buffer desired (nBufferSize) has been passed to the function, which can be determined before hand. */

/* Creating a temporary buffer to hold the values one need to return and filling that buffer. fill buffer, this buffer can be filled some other ways too, reading from a file or reading from some hardware or whatever  here it has been filled from values 1 – 255*/

     BYTE *pBuffer = (BYTE *) calloc ( nBufferSize, sizeof (BYTE))
     for ( int i = 0 ;  i < nBufferSize; i++ ) pBuffer [i] = ( i + 1 ) % 256 ; 

     /* Creating a variant to return, which can hold the safe array of bytes .*/

     SAFEARRAY FAR* psa ;
     psa = SafeArrayCreateVector (VT_UI1, 0, nBufferSize); 
      if(psa == NULL)    return –1 ;  
      
     /* Copying the temporary buffer to the variant to return */

     VariantInit ( pVar ) ;

     pVar->parray = psa;
     pVar->vt = VT_ARRAY | VT_UI1 ; 
     memcpy ( pVar->parray->pvData , pBuffer, nBufferSize * sizeof (BYTE) ) ;  

    /* free the temporary buffer */

    free ( pBuffer ) ;

    /* return the number of elements  this variant hold. */
    return nBufferSize;

}

as it can be seen from the above code, it involves following steps.

  1. Allocating the desired temporary buffer space to hold the array of bytes, and filling that array with values one need to return.
  2. creating a safe array of the desired (VT_UI1) type.
  3. copying the temporary buffer to the safe array of the variant.
  4. free the memory allocated for temporary buffer. (avoid memory leaks).

Fetching a array of bytes from a variant.

BYTE * GetByteBufferFromVariant (int nBufferSize, VARIANT var )
{

    /* here size of buffer desired (nBufferSize) has been passed to the function, which can help us determine the size of the buffer required. */

    BYTE *pBuffer = (BYTE *) calloc ( nBufferSize, sizeof (BYTE)) ;  
    memcpy ( pBuffer, var.parray->pvData, nBufferSize * sizeof (BYTE) ) ;  
    return pBuffer ;

}

In  a similar fashion array of some other simple types can be  packed in a variant. the following table describes the data types for different simple types.

S.N. Data Type Value of vt
1 BYTE VT_ARRAY | VT_UI1
2 SHORT (2 byte signed integer ) VT_ARRAY | VT_I2
3 USHORT (2 byte unsigned integer ) VT_ARRAY | VT_UI2
4 INT (4 byte signed integer ) VT_ARRAY | VT_I4
5 UINT (4 byte unsigned integer ) VT_ARRAY | VT_UI4
6 FLOAT ( 4 byte float value) VT_ARRAY|VT_R4
7 DOUBLE (8 byte double value) VT_ARRAY|VT_R8

 

using the table above one can pack array of above defined types, to variant, these conversion is useful in middleware scenario. in the next article I will explain how to pack array of strings and array of structure inside a variant.

July 17, 2013 Posted by | CodeProject, MFC, Win32 | , , | Leave a comment

Browsing Folders (VC++ 6.0)

while working with MFC there are dialog boxes for opening files, but no dialog box to browse  a folder, the function below just does the same. code snippet below shows how one can open a folder browsing dialog box.

BOOL BrowseFolder (

HWND hWnd,                // Handle to the calling window.

LPSTR pszDisplayName,      // Title of the dialog box

BOOL bEdit,                // whether edit box is allowed

LPSTR szPath )             // starting path 

{

// TODO: Add your control notification handler code here

LPMALLOC pMalloc;

BOOL bResult = FALSE ;

/* Gets the Shell’s default allocator */

if (::SHGetMalloc(&pMalloc) == NOERROR)

{

BROWSEINFO bi;

LPITEMIDLIST pidl ;

//Get help on BROWSEINFO struct – it’s got all the bit settings.

bi.hwndOwner = hWnd ;

bi.pidlRoot = NULL ;

bi.pszDisplayName = szPath ;

bi.lpszTitle = pszDisplayName ;

//bi.ulFlags = BIF_RETURNFSANCESTORS | BIF_RETURNONLYFSDIRS |BIF_EDITBOX ;

bi.ulFlags = BIF_RETURNONLYFSDIRS |BIF_VALIDATE ;

// whether you want to display an edit box

if ( bEdit ) bi.ulFlags = bi.ulFlags | BIF_EDITBOX ;

bi.lpfn = NULL;   // No call back function

bi.lParam = 0;    // No Extra parameter

// This next call issues the dialog box.

if ((pidl = ::SHBrowseForFolder(&bi)) != NULL)

{

if (::SHGetPathFromIDList(pidl, szPath))

{

// At this point pszBuffer contains the selected path 

bResult = TRUE ;

}

// Free the PIDL allocated by SHBrowseForFolder.

pMalloc->Free(pidl);

}

// Release the shell’s allocator.

pMalloc->Release();

}

return bResult ;

}

Calling the function.

TCHAR szPath [255] ;
_tcscpy ( szPath, "C:\\" ) ;
BOOL bSuccess =
BrowseFolder (

this->m_hWnd, "Broswe Source Folder", FALSE, szPath ) ;
if ( bSuccess )
{

  // szPath contains the selected folder.
}

July 14, 2013 Posted by | CodeProject, Win32 | , | Leave a comment