语言
现在阅读
VB6 and VBA: Creating a variable and an array that points to same data
0

VB6 and VBA: Creating a variable and an array that points to same data

由 ultracpy2018年1月26日

Introduction

The idea started after
I’ve read the nice article by Sam about parsing strings into JSON objects. Sam
decided to copy the string characters into an array of integers where is
becomes much faster to read and parse data. I was surprised by the performance gain that
we get even with the time wasted in copying data.

The problem was when
we have very large string that we have to buffer. In this case we have to
create an array of same size and duplicate the string data inside it. This will
consume a lot of memory and time to copy data.

Solution:

My idea is to create
an array that shares its data with the string. In other words the first integer
of the array is exactly the first character of the string and changing any of
them will change the other.

<o:p />

In this scenario we only need to create a small
array (one or two elements) and then change its underlying length and data to
point to the string length and characters.

<o:p />

Background

Both Visual Basic 6
and Visual Basic for Applications are built over COM structures and all of its
data types are Automation data types. So most of the variables are pointers to
COM structures.

<o:p />

Here we are interested
in two data types: Strings and Arrays:<o:p />

1.
String:

<o:p />

Strings are saved as Unicode
characters inside BSTR structure. We don’t need to Know about BSTR here. It
is enough to know that we can get a pointer to the character sequence of the
string by the famous undocumented function ‘StrPtr ()’.<o:p />

2.
Array:

<o:p />

Here the situation is
more complicated, because we need to change the underlying structure of the
array (both length and data pointer).<o:p />

Arrays are saved in a
SAFEARRAY structure which is defined as:

typedef struct tagSAFEARRAY {
    USHORT        cDims;        //The number of dimensions.
    USHORT        Features;    //Flags.
    ULONG        cbElements;    //The size of a single element.
    ULONG        cLocks;        //Number of locks.
    PVOID        pvData;        //The data.
    SAFEARRAYBOUND    rgsabound[1];    //one bound for each dimention.
} SAFEARRAY, *LPSAFEARRAY 

So when we get the safe array pointer we have to change the data pointer of the array (pvData) which is located after 12 bytes from the beginning of the SAFEARRAY structure.

We also need to change the array size which saved in ‘rgsabound’ member to reflect the size of the string. SAFEARRAYBOUND structure is defined as:

typedef struct tagSAFEARRAYBOUND {
    ULONG    cElements;    //The number of elements in the dimension.
    ULONG    lLbound;    //The lower bound of the dimension.
} SAFEARRAYBOUND, *LPSAFEARRAYBOUND;

So we have to change
cElements (which is located after 16 bytes from the beginning of the SAFEARRAY
structure) to reflect the number of elements we need for our array.

<o:p />

Note1: If you want to change the lower bound of your
array you can also change lLbound after 20 bytes from the beginning of the
SAFEARRAY structure.<o:p />

Note2: Be careful if you want to create a
multidimensional array. In this case every dimension will have a single
SAFEARRAYBOUND elements stored in rgsabound.

How to point to SAFEARRAY structure:

It is easy to get a pointer to any object in VB6 or VBA by direct using of VarPtr () function. This is true for everything except for arrays!! If you try to pass an array to that function you will get an error.

To solve this problem we have to create a new declaration for the VarPtr () function that takes an array as parameter. The problem about this way is that we need to change our declaration if we changed our VB version…. The following code describes how to do that:

'For VB6 users:
Private Declare Function VarPtrArray Lib "msvbvm60.dll" Alias "VarPtr" (var () As Any) As Long
'For VB5 users:
Private Declare Function VarPtrArray Lib "msvbvm50.dll" Alias "VarPtr" (var () As Any) As Long
'For VBA users with Office 2010+:
Private Declare Function VarPtrArray Lib "VBE7" Alias "VarPtr" (var () As Any) As Long
'For VBA users with Office before 2010:
Private Declare Function VarPtrArray Lib "VBE6" Alias "VarPtr" (var () As Any) As Long 

How to apply this method:

1. Get the string:

If you have a small string, please don’t waste your time by using this complicated method:

Dim S as string, Count as long
S = "Some large string"
Count = Len(s)  

2. Create a buffer array:

We will create a very small array of integers to create the underlying SAFEARRAY data:

Dim Buffer (1) as Integer ‘Two elements array 

3. Get the SAFEARRAY pointer:

As we said before, the VarPtrArray function will return a pointer to a pointer to the SAFEARRAY structure, for that we used a CopyMemoryToAny function to get the pointer value:

Private Declare Sub CopyMemoryToAny Lib "kernel32.dll" Alias "RtlMoveMemory"(ByRef Destination As Any, ByVal Source As Long, ByVal length As Long) 
Dim pArray as Long, pSafeArray as Long
pArray = VarPtrArray(Buffer ())
CopyMemoryToAny pSafeArray, pArray, 4  

4. Backup the SAFEARRAY original data:

It is very important to back up the old values of the SAFEARRAY structure and then restore them before VB clean this variable. If we fail in this the original values will remain uncleared and we may get into memory leakage problems:

Dim pOldData as Long
CopyMemoryToAny pOldData, pSafeArray + 12, 4 

5. Change to SAFEARRAY data to the string data:

Here we have to change the data pointer of the safe array (located after 12 bytes from the beginning) and then number of elements (located after 16 bytes):

Private Declare Sub CopyAnyToMemory Lib "kernel32.dll" Alias "RtlMoveMemory"(ByVal Destination As Long, ByRef Source As Any, ByVal length As Long)
CopyAnyToMemory pSafeArray + 12, StrPtr (S), 4
CopyAnyToMemory pSafeArray + 16, Count, 4 

6. Do your work:

Now you can start manipulating, parsing or even changing string data without changing its size. Please be careful not to erase your string variable or array data. This may cause unexpected behavior.

7. Restore the original SAFEARRAY data:

Now we can finish our work by restoring the old values of SAFEARRAY structure, so when VB start to clear unused data it will clear the right data of the array:

CopyAnyToMemory pSafeArray + 12, pOldData, 4
CopyAnyToMemory pSafeArray + 16, 2, 4 

Alternatives:

We can get direct access to string data also by using the undocumented function GetMem2 which can be declared as:

‘For VB6 users
Private Declare Sub GetInteger Lib "MSVBVM60.dll" Alias "GetMem2" (ByRef Src As Any, ByRef Dst As Integer)
‘For VB5 users
Private Declare Sub GetInteger Lib "MSVBVM50.dll" Alias "GetMem2" (ByRef Src As Any, ByRef Dst As Integer) 

There are two problems about these functions:
1. They are not available for VBA users.
2. They are slower than direct access by arrays which we described earlier, because there is little overhead when calling dll imported functions.

Example:

In the attached sample we created a 20 MB string then we started to count the numeric characters inside it using three five methods:

1. By using Mid$() function (String Access).
2. By creating a buffer array of integers and copy the string data into it (Array access).
3. By using GetMem2 () function (Memory Access).
4. By creating an array of integers that point to the string data (Direct access).
5. By using a dll function written using C++ 2012 (C++ API access), this will work only from the exe file not from the IDE unless you have changed it declaration.

The performance gain against String access was really huge (30 times faster in EXE), where the speed’s gain between Array and Direct access was very small…. But actually there was a memory’s save of about 40 MB because we did not use a copy of data to parse the string. On the other hand, Memory access gave us a good performance with no memory over load and less complicated code, it is about 2-3 times slower than the last two methods. And for sure the out performance winner is C++ API access which is the fastest, but not much more than direct access method.

Caution!!! :

There is a strange behavior in this method….. When you run your application in inside the IDE it will work perfectly, but when you compile it, it will raise a ”Subscript out of range (Error 9)”, unless you compile it with removing bound array checks. You can remove it when you compile the project by pressing Option button, then go to compile tab and press Advanced Optimizations button then check Remove array bound checks.

More to do……

In this article we explained how to parse a string by direct access to its characters using an array of integers. You can also change its data if you did not change its length, for example formatting the strings to change some characters to upper case. You can also use the same concept to access any type of memory data by using arrays, For example GDI+ bitmaps data through there Lockbits function.

出处:https://www.codeproject.com/Articles/729235/VB-and-VBA-Array-Buffering

关于作者
ultracpy
评论

你必须 登录 提交评论