Victory Fernandes and I have decided to make TKinect (v2) available to the public as-is, and give the public access to the development working copy as it evolves.
This means that the SVN repository you can check out today (details below) is the exact working copy Victory, Embarcadero and I are committing to every time we make any changes or additions.
At this stage, you do not need to install any packages, as what you will find is a single unit Kinect.API.pas containing a mostly-complete translation of the Kinect SDK headers. This means that, for the moment, you can experiment with the Kinect SDK in Delphi by some simple translation of the C/C++ documentation on the Kinect SDK website!
Two demo projects are provided, but the one you’ll most likely want to play with is TK2CameraTest.exe, which provides skeletal mapping of up to 6 people in the Kinect’s field of view onto a Bitmap Canvas.
The Kinect.API.pas unit should work with all versions of Delphi, and even FreePascal. UPDATE: Actually it’ll work with Delphi 2006 and above due to the use of methods in a record type!
Please note that the Kinect SDK is only available for Win32 and Win64, so do not expect this to work on MacOSX.
The TKinect classes will be added in the very near future, but we cannot develop this until we figure out what’s stopping us from accessing the raw Depth and RGB frame data via the API.
Contributions to our efforts (in the form of research and development) are greatly appreciated! You may submit code changes either via comment on this blog, or by e-mail (see Kinect.API.pas for the e-mail address).
How to get TKinect v2
Use an SVN Client (or the Integrated SVN client in XE/XE2).
Repository Address: http://svn.lakraven.com:81/svn/TKinect2
Username and Password (both): anon
You can either load Grp_TKinect_XE2.groupproj if you are a Delphi XE2 user, or just load \Demos\2 – Camera Test\TK2CameraTest.dpr (or .dproj) for all other versions of Delphi.
Please let me know if you experience any problems.
Troubleshooting
Q: I am running TK2CameraTest.exe but it is not picking up my skeleton
A: You need to be a good few meters away from the Kinect controller, and within its field of view! Please try testing in a nice open space where you can reorient yourself to see if the Kinect picks you up at certain positions and distances!
Q: I’m getting some sort of error immediately after loading TK2CameraTest.exe
A: Do you have the Kinect SDK installed? Try running one of its two bundled pre-compiled demos to see if they are working!
Q: I was using TKinect (the original one) and I can’t run the new demos
A: You need the OFFICIAL Microsoft Kinect SDK! TKinect v1 used the NUI SDK (an unofficial 3rd-party driver).
Acknowledgements
I just want to emphasize that the header translation was a team effort, made possible with the invaluable assistance of Victory Fernandes (whom is now my partner on the TKinect project) and Embarcadero Technologies (particularly Allen Bauer).
December 7, 2011 at 3:05 am
The following is my test result:
———————————————————-
- TK2CameraTest not always works, the tracked skeleton is not smooth and looks unstable. (On the other hand, the Skeletal Viewer of Kinect SDK Beta 2 is stable and smooth with the same computer, I haven’t idea about the reason)
- Cannot compile TK2SimpleAPITest because the compiler cannot resolve “Windows” and “Messages”, unless I change them to “Winapi.Windows” and “Winapi.Messages” (but I don’t need to do that with TK2CameraTest ???). Only the “Cam Down” and “Cam Up” buttons work finally.
Delphi Version: XE2 Update 2
Kinect SDK Version: Kinect SDK Beta 2
December 7, 2011 at 1:11 pm
The reason you cannot compile TK2SimpleAPITest is that it was made with XE, and as such doesn’t have the define in the dproj file to state whether it uses VCL or FireMonkey. If you add a VCL form to the project in XE2, it’ll ask if you want to set the project to VCL… saying “yes” will immute the need to define the scope before the unit names.
I am aware of some “glitches” in the Skeleton demo (TK2CameraTest), which we’re trying to figure out… it’s part of the reason I’ve made it public (since more heads will hopefully locate and resolve this bug quicker).
Cheers for giving your results, and especially for letting me know which version of XE2 and the SDK you’re using.
It’s also possible I need to update my copy of the SDK (along with Victory and the guys at Embarcadero) since I believe we’ve developed this on Beta 1!
December 8, 2011 at 2:36 am
Thanks
It is really cool, especially in Delphi.
December 9, 2011 at 4:56 am
You’re welcome!
December 7, 2011 at 7:26 am
Thank you very much for this interesting submission.
Keep up the good work.
December 7, 2011 at 1:08 pm
Thanks
December 8, 2011 at 12:09 am
Hi Simon!
As I read in your profile, you are a specialist in automation. Any ideas about the uses of the kinect in the automation world?
Good work and thanks
December 9, 2011 at 5:01 am
I should emphasize first that I am a specialist of Process Automation, not “automated manufacturing” (though the two are at a software level fairly similar in many respects).
Since the Kinect gives you a “3D map” of the environment within its frame, there are an infinite number of potential applications for the technology in all areas and industries. For a start, if you have a production line mass-producing a product of a specific shape, you could provide within your software the “geometry” of this product, and have the software automatically assess if each item looks the way it should. You could further determine if there are any timing issues on the assembly equipment by having the software measure the gap between each item. If the gap starts to grow or shrink beyond a known tolerence, it could alert the operator(s) that there may be a problem.
Hell… with enough math and effort, you could even use the Kinect to assess threats to the workforce (such as a forklift travelling too quickly with one or more people or objects in its path), and have the system alert the appropriate individual(s) of this.
The limitation really is your imagination (and the skills of the developers implementing those ideas).
December 8, 2011 at 9:47 pm
…both projects don’t work for me:
TK2CameraTest -> ntdll.dll error
TK2SimpleAPITest -> only the cam up-down feature works fine, if I try to take an image, then I get a MSRKINECTNUI.dll error
Delphi XE2 Update 2, Windows 7 64, Kinect SDK beta 2
December 9, 2011 at 4:56 am
Beta 2? I wasn’t aware there was a Beta 2! It would appear those of us developing TKinect need to update the SDK.
I’ll get back to you shortly!
December 9, 2011 at 4:12 pm
Since I just wrote my own Kinect SDK Delphi conversion, I also took the time to fix yours.
The camera is now working, and I implemented the INuiInterface, and I updated it for Beta 2, and I added a function for converting Kinect HResults to strings.
Microsoft made the messiest API I have ever seen for this. It mixes cdecl, stdcall, and thiscall calling conventions (thiscall isn’t supported by Delphi, or anything except visual C++). It has Interfaces that don’t descend from IUnknown (again, not supported by Delphi). It has one function where NUI is all in capitals. It has a bunch of functions that aren’t declared in the header files or documented anywhere that it still expects you to call.
I faked the interfaces with Delphi classes and virtual methods, and faked the thiscall calling convention with the Register calling convention and added static methods that rearrange the order of the parameters.
I changed parameters from pointers to “var” and “out” where appropriate (and changed their hungarian prefix accordingly).
I just need to work out how to submit it to you…
December 9, 2011 at 6:32 pm
Hi Carl…. incredible work there!
Couldn’t agree more that this API is amongst the messiest I’ve ever encountered! I was not aware that it was mixing cdecl, stdcall and thiscall, this certainly explains a lot of the glitches and bugs we’ve been having! I never even thought to LOOK as I’d just presumed they would have used one calling convention throughout (like any sane developer would).
I’ve committed your patch and sent you an e-mail! Thank you very much for this
December 11, 2011 at 2:00 am
I Posted the news at the FB Delphi Brazil Community!!!
December 11, 2011 at 2:04 am
Thanks, Victory! Well done on the presentation at the conference in Brazil
December 12, 2011 at 11:08 pm
Simon, is this version on the SVN already patched with the Carl’s contribution? He pointed up very serious issues. Blame MS.
December 12, 2011 at 11:12 pm
Yes, I published another article to announce the update.
THE SVN IS ALWAYS UP TO DATE WITH THE LATEST DEVELOPMENT CODE
January 16, 2012 at 10:52 pm
Simon,
Keep up the GREAT work. Do you think the Kinect component will ship with new Delphi XE3?
I’m a Old use of delphi and I’m glad Embarcadero is starting to upgrade Delphi. I’m hoping to see more upgrades like Kinect, parallel programming, and upgrade to some old component like media player to save wave file to a MP3 format.
Thankyou for all your great programming, and Helping Delphi become a stronger product!!!!
Troy
January 17, 2012 at 3:42 pm
I’m hoping TKinect will be included in the next version of Delphi as standard… though it’s name is about to change (because it’s no longer just for the Kinect controller… article soon to explain that more).
Delphi has undergone a LOT of changes since Version 7… most notably things like Unicode and now with XE2 cross-platform & Win64!
Generally-speaking it is getting better now with each successive version… and that’s a step in the right direction, for sure!
February 14, 2012 at 7:56 am
I am tryin to download TKinect2 with Tortoise and I can’t put the user and pwd. I have always a “forbidden” error.
Thank you.
February 14, 2012 at 10:54 am
I’ve never encountered this issue! Always when checking out a new repository which requires a username & password, it prompts for those details after entering the URL and pressing “Checkout”.
This is one where, I’m affraid, you’ll have to Google the solution or consult TortoiseSVN’s FAQs and documentation.
March 5, 2012 at 11:58 pm
Hi Simon,
I have some trouble with the Camera Test demo.
The program can’t find the msrkinectnui.dll file
I installed the microsoft sdk (V1.0)
Do you have any suggestion what’s going wrong?
Regards,
Frank
March 6, 2012 at 8:12 am
Using a Kinect for Xbox controller by any chance?
March 6, 2012 at 10:22 pm
Oops… yes.
March 8, 2012 at 11:16 am
Hi There,
is there any chance that Tkinect2 can work with Kinect for Xbox controller ?
March 8, 2012 at 12:54 pm
Not since the final release of the Kinect SDK for Windows, in which they deliberately screwed all of us whom generated the demand for official Kinect support with Windows by making it a requirement that you spend MUCH more money on the exact same controller (only without the “XBOX360″ branding on the front).
I refuse to purchase another Kinect controller, especially since it doesn’t have a single enhancement over the original.
Microsoft are bastards, and I genuinely hope that NOBODY buys another Kinect from them ever.
TKinect has now become GeNUI, and while providing a degree of support for the official Windows Kinect SDK, it focuses more on PURE PrimeSense controllers such as the ASUS Xtion Pro Live. These controllers are much cheaper than the Kinect for Windows crap, but also superior in EVERY SINGLE WAY! This is where the future resides, not in Microsoft’s latest money scam!
March 8, 2012 at 3:04 pm
ohh so i can’t do anything with my xbox 360 Kinect controller in delphi ? :\
March 8, 2012 at 3:31 pm
With the official Windows Kinect SDK, it appears as though you cannot (I haven’t actually gotten mine to work with it since the Feb 1st release from Microliars). You could, however, use the original TKinect source with the Code Labs NUI SDK (still available to download), but this obviously wouldn’t work alongside the official Microfuckers’ SDK.
March 8, 2012 at 3:41 pm
i see… ok thanks a lot for your help!
March 8, 2012 at 3:59 pm
how can it be working with Microsoft’s Demos ?
March 8, 2012 at 5:24 pm
Really? Mine won’t work with even the demos since I installed the final version of the SDK! I cannot get it to do a godamn thing but sit there blinking a green LED at me inanely, mocking me for the hard work I put in (along with many other PC devs out there) to generate interest from the PC community.
March 9, 2012 at 8:18 am
It’s very strange, because Microsoft wrote in their website that you can use your Xbox 360, it is less recommended but possible.
Besides there are many developers that have done well with Xbox 360 such as http://www.evoluce.com
I downloaded their SDK and it works very nice with Xbox 360!
but no Delphi support, only C#
March 9, 2012 at 8:36 am
I wonder if they’ve changed some godamn calling conventions AGAIN. I’ll take another careful look through the headers soon as I have time.
March 9, 2012 at 10:16 pm
Hi Stuart,
You have done a great job with Tkinect2 in combination with SDK 2.0: supporting multiple kinects, fast code, the basis for a lot of applications.
Hopefully you will succeed to get the headers right!
Good luck!
March 20, 2012 at 4:52 pm
I have looked to the headers and I have found the next differences
function MSR_NUIGetDeviceCount(out refCount: Integer): HRESULT; cdecl; external KINECT_DLL;
function MSR_NuiCreateInstanceByIndex(const Index: Integer; out refInstance: INuiInstance): HRESULT; cdecl; external KINECT_DLL;
procedure MSR_NuiDestroyInstance(Instance: INuiInstance); cdecl; external KINECT_DLL;
is changed in:
function NuiGetSensorCount(out refCount: Integer): HRESULT; cdecl; external KINECT_DLL;
function NuiCreateSensorByIndex(const Index: Integer; out refInstance: INuiInstance): HRESULT; stdcall; external KINECT_DLL;
procedure NuiDestroyInstance(Instance: INuiInstance); cdecl; external KINECT_DLL;
Then I looked to the function NuiImageStreamOpen, but this call is incorrect. I thought:
: _NuiImageStreamOpen(//EDX: Integer;
i: INuiInstance;
eImageType: NUI_IMAGE_TYPE;
eResolution: NUI_IMAGE_RESOLUTION;
dwImageFrameFlags_NotUsed: DWORD;
dwFrameLimit: DWORD;
hNextFrameEvent: THandle;
out refhStreamHandle: THandle
): HRESULT; virtual; {Register; }stdcall; abstract;
Instead of :
function _NuiImageStreamOpen(EDX: Integer; i: INuiInstance;
out refhStreamHandle: THandle;
hNextFrameEvent: THandle;
dwFrameLimit: DWORD;
dwImageFrameFlags_NotUsed: DWORD;
eResolution: NUI_IMAGE_RESOLUTION;
eImageType: NUI_IMAGE_TYPE): HRESULT; virtual; Register; abstract;
The order of parameters is different and also the convention call.
Is this right? Do you have suggestions for the last function(NuiImageStreamOpen)?
April 21, 2012 at 7:15 pm
Cheers for letting me know.
Please be advised that this project has changed dramatically… and even has a new name (GeNUI). The new project doesn’t focus exclusively on the Kinect controller (you can thank Microsoft’s lies and extortionate act of forcing everyone to buy another “Kinect for Windows” controller – identical to the original Xbox one, with literally NOTHING NEW AT ALL – to unlock the 45-80cm tracking range, which is essential for any real-world use of the Kinect hardware and they’re charging TWICE the price for it). We now focus mainly on the raw PrimeSense hardware (such as the ASUS Xtion PRO Live), with the Kinect as a SECONDARY concern.
March 9, 2012 at 8:37 am
Oops
here is the translate:
Could be because I have a new Kinect? I bought it yesterday …
March 9, 2012 at 8:46 am
Well my Kinect is well over a year old, now, and it won’t work with the release version of the Kinect SDK either.
March 11, 2012 at 1:48 pm
hi again
Just so you know, maybe this will help somehow.
Once I installed the Microsoft Visual C# 2010 Express
i can run your example “2 – Camera Test” with my Xbox’s controller without any dll missing errors
Regards,
yoni.
March 11, 2012 at 4:01 pm
Ah, so it would seem as though the Redistributable package or Visual Studio itself is required to use the SDK?
March 11, 2012 at 4:22 pm
i have no idea all i know is once i installed it everything is working great!!
and with the xbox controller, i think it added some dlls and stuff.
btw great job for Tkinect!
March 11, 2012 at 5:02 pm
Cheers! Wait until you see GeNUI
March 12, 2012 at 3:47 pm
Wow there is something you can do with the example: 2 – Camera test ? It kills the processor every time I start it!
March 12, 2012 at 3:59 pm
Annoyingly there’s nothing I can do about the piss-poor state of the Kinect SDK… I will say it doesn’t kill my processors, but then again I cannot get my Kinect to work at all on Windows now.
March 14, 2012 at 2:59 pm
ok, so I checked it, you don’t have to install visual studio
all you need is to copy these 2 dll :
MSRKinectNui.dll
KinectSQM.dll
to “C:\Windows” directory
And your Xbox controller will works just fine
i uploaded it to my server so you can download it, if you want.
http://novu.co.il/KinectDlls.rar
best regards,
yoni.
March 14, 2012 at 5:01 pm
Cheers
March 25, 2012 at 2:59 pm
hi
I wanted to ask you, Is it possible to detect if a person makes a fist?
March 25, 2012 at 11:03 pm
If it’s tracking each individual finger PRIOR to the operator making a fist… then it’s easy enough!
March 26, 2012 at 8:57 am
I could not detect any fingers :\
can you post some code, example or anything that can help me ?
April 21, 2012 at 7:00 pm
There is a project provided as part of the OLD TKinect SVN repository (which is still available on http://svn.lakraven.com:81 using “anon” for username and password) which includes a simple algorithm to detect the human hand and fingertips.
You would need to make the necessary changes to support the TKinect v2 codebase!
We haven’t yet released the new GeNUI suite (it’s still in early development due to all of us working on it having other more pressing engagements… I.E. the stuff that pays all of our bills), but this will absolutely include hand and finger tracking as standard!
April 29, 2012 at 9:51 am
ok thanks a lot!!