ofx.mod - OpenFrameworks

BlitzMax Forums/Brucey's Modules/ofx.mod - OpenFrameworks

Brucey(Posted 2009) [#1]
Another weekend... another module.

This one has been a lot of fun so far - mostly because I've spent too much time "playing" with it, than finished the coding... oh well :-p

OpenFrameworks is a framework designed for "creative coding".
If you are familiar with Processing, the Java framework, then you'll have an idea of what this is about.

It is a cross-platform OpenGL-based framework.
It does images, drawing, video, sound, comms, and various other things... and is expandable via addons. (like OpenCV)

With the expandibility in mind, I've created it in its own namespace "ofx", rather than "bah", since it makes more sense once new modules begin to appear.

I've only worked on the Mac and Windows ports for now. Everything appears to build okay.

The API is very incomplete at the moment - I'm still pondering how to do the events/keys/mouse handling. (If anyone wants to help out... I'm happy for the assistance)

I've knocked together a couple of examples. They are mostly complete, excepting the key/mouse stuff.
I've also built in rudimentary Max2D integration, which means you can mix and match both OpenFrameworks and Max2D. In fact, there is no reason you couldn't use MiniB3D either.

The project page is at : http://code.google.com/p/ofxmax/
Currently only available via the SVN repository at : http://ofxmax.googlecode.com/svn/trunk/

One thing to note... It's rather BIG (over 110meg).. so it may take a while to download the first time. (but there's a price to pay for having lots of cool toys).

They are looking to release it as LGPL, so at some point I'll be changing the current "compile in the source" method, to link to a DLL instead - which will solve any potential license issues. But I've not done that yet, as it's quicker to throw it all together this way and have a basic working version.
Before a "proper" release, this would change, of course.

Oh... and the external requirements :

BaH.FMOD (I don't make the rules about what audio engine to use with the framework)

Installation :

The base module ofx.mod should be placed as such :
Examples can currently be found in core.mod/examples and ofxopencv.mod/examples.

As ever, comments and whatnot always appreciated.


xlsior(Posted 2009) [#2]
BaH.FMOD (I don't make the rules about what audio engine to use with the framework)

Just curious -- Does it also require fmod if you aren't using any of the sound commands?

Brucey(Posted 2009) [#3]
That appears to be the case - I don't have control over that though.

Hopefully they'll split things up a bit as the project moves forward.
They've already pulled out GLUT as a requirement, which was good for me :-)

Brucey(Posted 2009) [#4]
Now supporting Linux :-)

Requires a bunch of libraries, and some luck... but it's running fine here. (also shows some inherent problems with BMK.... but you probably don't care about that).

On Ubuntu, you probably want to run this :
apt-get install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libavcodec-dev libavformat-dev libavutil-dev libswscale-dev freeglut3-dev libasound2-dev libxmu-dev libraw1394-dev
(lifted from openFrameworks)

That should get you required headers and libraries.

Otherwise, call back here and we'll see if we can't work out what's wrong.

DavidDC(Posted 2009) [#5]
Builds and runs nicely on XP. Fun, fun, fun! Thanks Brucey :-)

[edit] btw your "hand" example (webcam contour finder) runs at 1195 FPS on my 2006 iMac. I was pretty amazed at the speed.

wmaass(Posted 2009) [#6]
Got it working! Looks very promising Brucey. Might provide another option on tracking for my putting project. If I get to the point to where I can use this this to measure the speed and angle a rolling golf ball it would be a better solution over the Phigets or the WiiMote solution I came up with.

Brucey(Posted 2009) [#7]
You should be able to see the tracking working with the opencv example.

I've still got a lot of stuff implement but even from the little I've done so far, I can see it has a lot of potential for some interesting projects.

Also, if you find any interesting addons for OpenFrameworks, let me know :-)

wmaass(Posted 2009) [#8]
Yep, I rolled a golf ball through the view area and it tracked great. I just need to know the blob center in at least 2 frames and I am all set! I'll keep my eyes open for addons.

wmaass(Posted 2009) [#9]
Oh and all of this makes possible Multitouch Tables in Bmax, along the lines of NuiGroup/TouchLib, no?


Brucey(Posted 2009) [#10]
I'll work on the rest of the opencv API tonight then... there's direct access to the blobs through it - rather than the combined set that the example shows.

wmaass(Posted 2009) [#11]

Brucey(Posted 2009) [#12]
Added blobs and a blob iterator (see the example on how to get access to them through the contour finder).
It uses BlitzMax's enumerator thing... so you can process the list of blobs via an EachIn loop.

I've still to implement access to the points list in the blob itself - it took me a lot longer to get the iterator working that it should have... C++ Pah!

makes possible Multitouch Tables in Bmax

Possibly. I suppose it depends how the driver presents the multi-touch data to the application. (maybe someone has already written some multi-touch addon)

wmaass(Posted 2009) [#13]
Nice, getting ready to lock myself in my office for a while and play with this.

Armitage 1982(Posted 2009) [#14]
Nice !
I always wanter to play a bit with Process.

Wonder if things like Augmented Reality are easy to do with this kind of framework ?
Maybe the toy I was waiting for :)

Can't wait to see the end of this one !

byo(Posted 2009) [#15]
Cool! Thank you for this.
I think I'm gonna lock myself in my room as well. :D

wmaass(Posted 2009) [#16]
Well Brucey thanks to your recent update I am able to measure the speed of a rolling ball. The only drawback is that the ball can't be moving more than a meter per second or so, else I don't get any blobs detected. I theorize that once you implement a more direct access to the blobs (if such a thing exists) it will be better since you won't be relying on the contour finder to get at them. Thanks again for your work on this, it really is very exciting.

plash(Posted 2009) [#17]
Strange error.. I'm using Brucey's optimized bmk_ng under windows, and just got the code from the repo.

(Typical MaxIDE Ctrl+D)
format_code('Building Modules
Build Error: failed to compile (1) C:/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/app/ofAppRunner.cpp
??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????d̉ec: no input files
Build Error: failed to compile (1) C:/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/app/ofAppRunner.cpp
Process complete')

EDIT: I switched back to the original BMK so I could build and play with it.

Brucey(Posted 2009) [#18]
That's interesting, because DavidDC had that error... but when he built it a second time it worked fine.
On the same file, no less... so I'm not sure what is up with that.

DavidDC(Posted 2009) [#19]
Yes I had exactly the same error with bmk NG. Used Ctrl +D as well. Vanished on second attempt. Didn't do a thing other than press Ctrl +D twice.

wmaass(Posted 2009) [#20]
Hmm, noticed some updates, what have you been up to Brucey :)

byo(Posted 2009) [#21]
Brucey, here's an addon that would be great to have:

ofxFiducialFinder finds and tracks fiducials from video source.

ofxFiducial has... width recognition, Jitter filtering, frame flashing on/off filtering, rotation speed, rotation acceleration, motion speed, motion acceleration, motion speed x, motion speed y, a draw and drawCorners feature, getter functions which include getangle in radian and degree(compensated), = operator overloading ( I don't think any more is needed but if requested I will), point inside fiducial helper function check, fiducial collision helper function check, get distance from fiducial helper function


Brucey(Posted 2009) [#22]

The top-left image shows the live tracking of "fiducial" objects.
The filled blue circle under the mouse pointer denotes the pointer location is inside a fiducial object.
In this example I was just pointing the camera at the screen, which was displaying a PDF of known fiducial symbols.

Very interesting :-)

Note : ofxFiducialFinder is GPL'd... (this seems to be quite common in this area of visualisation. On the other-hand, OpenCV itself is BSD).

Brucey(Posted 2009) [#23]
I've added ofxCvHaarFinder support.
The example shows face recognition - that is, recognising that there is a face in an image rather than recognising *who* is in the image.
The routines are a bit of a resource-hog, so you may find it runs slow-ish on your computer.
You could also test it by feeding it a photo which contains people's faces - it should be able to detect (and highlight) them.

(see haarfinder example in the ofxOpenCv module)

Fun fun...

byo(Posted 2009) [#24]
Very neat. Thank you. :)
*back to play with ofx*

wmaass(Posted 2009) [#25]
Oh, cool!

So far I've managed to do 2 things I've wanted to do:

- Track a rolling golf ball
- Use any surface such as a wall, floor etc as a touch screen of sorts

I'll post some examples once I mature them a bit more.

Brucey(Posted 2009) [#26]
@wmaas, openCV has some functions for tracking objects, which aren't currently implemented in openFrameworks. They may make tracking much easier to implement - given that they utilise all kinds of algorithms to do it.

But, since I've only started touching on the surface of this project, I imagine there's all kinds of cool stuff to use.

wmaass(Posted 2009) [#27]
I'm sure you are right. It would be interesting to see what I could do with more robust tracking functions, but I'm pleased with what I have been able to do so far with what you have implemented.

wmaass(Posted 2009) [#28]

I'm a bit new to the concept of Types and Methods. I would like to incorporate OpenFrameworks with the likes of LeadWerks Engine and MiniB3d. Would you be able to give me a few pointers as to how to structure the code? Typically everything goes on in a main loop but I am unsure how to reconcile the methods in OpenFrameworks with this.

wmaass(Posted 2009) [#29]
I've got several fun projects going with this but here is something fun I knocked together with this stuff.


Armitage 1982(Posted 2009) [#30]
Hi Brucey
What's the current statut of ofx module ?

Do you still plan to finish it or is this module ready enough to be consistent ?


Brucey(Posted 2009) [#31]
I've just checked in support for keyboard and mouse.

This is demonstrated in the movieplayer example :

I've decided not to convert keys to OF key mappings, since it's just an extra conversion that we don't really need, as we already have a working solution in the keycodes module. (unless someone prefers otherwise?)

Example (bmx):
Method keyPressed(key:Int)
Select key
Case KEY_F
frameByframe = Not frameByframe
Case KEY_0
End Select
End Method

Example (c++) :
void testApp::keyPressed (Int key){
Case 'f':
Case '0':
As you can see, I think it maps fairly well as it is.

wmaass(Posted 2009) [#32]

Armitage 1982(Posted 2009) [#33]
Nice change :)
I just read from here and there that KEY map is sometimes different from on platform to another.
And that's maybe true since only from Windows to Linux functions keys are different.
I can't say much about MacosX since I can only build the binaries and not really test this at regular speed.
I should really think about a MacMini with Leopard if I want to support that platform but it's still pretty expensive for me at the moment.

Brucey(Posted 2009) [#34]
Also checked in support for ofImage.

Added a small "Processing" example port to show some interesting image use of a lens effect :

Of course, ofImage has much the same functionality as TPixmap/TImage, but, since it's part of the framework...

wmaass(Posted 2009) [#35]
Keep it coming Brucey.

Armitage 1982(Posted 2009) [#36]
I don't know why but my webcam display in every example is up side down ?
Which is not the case with a regular webcam display program.

It wasn't the case few version before (I can't tell you exactly when but at least 3 months).

By the way Brucey, what can you tell us about the Module status ?
Do you exactly know what kind of ofx modules you are about to implement ?
Would be nice to know in order to read a bit about every modules features and see what's currently doable.

I'm still only playing with your examples for the moment.

MoviePlayer example is also reverse while the black and white dot matrix one is correctly displayed.

Brucey(Posted 2009) [#37]
I don't know why but my webcam display in every example is up side down

That's completely my fault.

Seems I lost my head when working on the ofImage support (which draws them upside down) and thought the best place to fix it would be in ofTexture - which it wasn't :-p

I'll undo that "fix" and try to get ofImage working properly instead.

Brucey(Posted 2009) [#38]
Okay, I've hacked ofImage to y-flip the FreeImage bitmap before loading it into the ofImage...
Funnily enough, I also do this for pixmaps that are loaded by BaH.FreeImage.

The included "processing" examples are quite interesting, I think. I've been porting some of these Java examples to openframeworks. Since openframeworks is meant to share some common ground with Processing, the porting process isn't too hard - although there is actually a lot more functionality in Processing. Hopefully this will change over time.

Armitage 1982(Posted 2009) [#39]
Nice :)

I didn't know that ofx is capable to reproduce Processing examples.

By more functionality over time you mean the official ofx framework roadmap or your ofx modules ?
Would be great to be able to do exactly the same Processing stuff with ofx !

I just wonder how fast ofx rendering is compared to Max2d.
If full realtime is still possible with game running in parallel

plash(Posted 2009) [#40]
Have the dependencies changed at all for Linux?

I'm getting these errors:
format_code('/media/system/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/video/ofUCUtils.cpp:19:22: error: avformat.h: No such file or directory
/media/system/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/video/ofUCUtils.cpp:20:21: error: swscale.h: No such file or directory
Build Error: failed to compile (1) /media/system/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/video/ofUCUtils.cpp')

Based on 'ofUCUtils.cpp', the includes have something to do with ffmpeg and Jaunty.

EDIT: As suggested by the comments above the includes, commenting the first two made it compile..
EDIT2: Had to get all the Poco libs because they were imported in ofx.core. Now it says it can't find '-lGLee', which I cannot find.
EDIT3: The second edit pertains to the example code, not the modules. The modules seem to build OK, but when trying to compile and run an example it freaks out about missing libraries.

Brucey(Posted 2009) [#41]
Have the dependencies changed at all for Linux?

Yeah, my fault. I'd fixed this in the previous update, but hadn't noted it anywhere ;-)

Noted in core.bmx now - so I'll remember for next time! (hopefully)

plash(Posted 2009) [#42]
Still can't get the example to work. I can't find Glee anywhere in Synaptic.

This is the error message:
format_code('/usr/bin/ld: cannot find -lGLee
collect2: ld returned 1 exit status
Build Error: Failed to link ~/blitzmax/mod/ofx.mod/ofxmsafluid.mod/examples/example_01')

wmaass(Posted 2009) [#43]
Brucey, got any wisdom on how to get this to play nice with MiniB3d or other engine?

Brucey(Posted 2009) [#44]
Since it's just a standard GL context it should play fine.

In fact, if you look at the ofxglmax2d module, you'll see how I've implemented it.

By default, openframeworks uses GLUT, but since we already have our own stuff in BlitzMax for dealing with GL, I thought it was more compatible for use to use the routines that are already available to us.

(I realise it's a bit bare in there still, but the most useful methods are implemented :-)

wmaass(Posted 2009) [#45]
Cool, I'll take a peak at ofxglmax2d and see what I can figure out.

Brucey(Posted 2009) [#46]
Still can't get the example to work. I can't find Glee anywhere in Synaptic.

I'll get there sometime.

Applied yet another "fix" to it which seems to be more in the right direction.

I'll take a peak at ofxglmax2d and see what I can figure out

Note that there are already a couple of functions posted somewhere that let you switch between 2D and 3D view that I think Klepto made. (possibly search for max2d in the minib3d forum)

wmaass(Posted 2009) [#47]
This looks promising...


wmaass(Posted 2009) [#48]

I updated to the latest, rebuilt everything. I had to go get bah.random to build core and now I'm getting this with my projects. Any idea?

C:/Program Files/BlitzMax/mod/bah.mod/random.mod/random.debug.win32.x86.a(random.bmx.debug.win32.x86.o): multiple definition of `__bb_random_random'
Warning: .drectve `/DEFAULTLIB:"LIBC" /DEFAULTLIB:"OLDNAMES" ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:uuid.lib ' unrecognized
Warning: .drectve `-defaultlib:uuid.lib ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Build Error: Failed to link C:\Program Files\BlitzMax\mod\ofx.mod\ofxopencv.mod\examples\myGui_7.debug.exe

plash(Posted 2009) [#49]
Applied yet another "fix" to it which seems to be more in the right direction.
Was r46 intended to work? You didn't add 'ofx.mod/core.mod/libs/GLee/lib/linux/libGLee.a' to the repository.

EDIT: Tossed the library in there from http://nuicode.com/projects/tbeta/repository/revisions/169/changes/trunk/tbeta/Linux/libs/glee/libGLee.a and it worked.

Brucey(Posted 2009) [#50]
multiple definition of `__bb_random_random

I suppose I could substitute Bah.Random for BRL.Random - although the official Random module is not consistent across platforms (tsk tsk).
It's part of the new perlin noise functionality - which I've implemented in BlitzMax, as opposed to C++.
Maybe I should split it out into a separate module - yeah... probably better, since it's not really part of the original core framework.

The other warnings, you get anyway. But they are just warnings.

Brucey(Posted 2009) [#51]
You didn't add...

Funnily enough, I added it to my project, then checked everything in.
However, I forgot (as I often do) that .a files are by default not imported by subversion, so it wasn't actually picked up by the commit, and didn't show up in the project as a change.
Quite annoying the .a thing. You have to specifically "add" them to the project.

Oh well... sorry for the long, dragged out, round about process.

My Linux is playing up, which I think doesn't help my temperament...

wmaass(Posted 2009) [#52]
No worries Brucey, I backed out to the last working version I had. And I think I made some progress on the 3D integration by using MiniB3D with Oddball's stuff at the post below:


Anyway, check it out.


Framework ofx.ofxopencv
Import ofx.ofxglmax2d
Import brl.glmax2d
Import "minib3d.bmx"

Graphics3D 1024,768

Global camera=CreateCamera()
Global light=CreateLight()
Global world=CreateSphere(12)

tex = LoadTexture("brick.jpg")
EntityTexture world,tex

Global myapp:TTestApp

PositionEntity camera,-5,0,-10 '***
PositionEntity world,0,0,0
PointEntity camera, world '***

myapp = New TTestApp

While Not KeyHit(KEY_ESCAPE) '***

TurnEntity world,0,1,0


'SetBlend AlphaBlend
'SetColor 255,0,0
'SetBlend SolidBlend






Type TTestApp Extends ofBaseApp

Field vidGrabber:ofVideoGrabber

Field colorImg:ofxCvColorImage

Field grayImage:ofxCvGrayscaleImage
Field grayBg:ofxCvGrayscaleImage
Field grayDiff:ofxCvGrayscaleImage

Field contourFinder:ofxCvContourFinder

Field learnBackground:Int
Field threshold:Int

Method setup()
vidGrabber = New ofVideoGrabber

colorImg = New ofxCvColorImage

grayImage = New ofxCvGrayscaleImage

grayBg = New ofxCvGrayscaleImage

grayDiff = New ofxCvGrayscaleImage

contourFinder = New ofxCvContourFinder

learnBackground = True
threshold = 80
End Method

Method update()

Local newFrame:Int = False

newFrame = vidGrabber.isFrameNew()

If newFrame Then

colorImg.setFromPixels(vidGrabber.getPixels(), 320,240)

If learnBackground = True Then
grayBg.copy(grayImage) ' copies the pixels from grayImage into grayBg
learnBackground = False
End If

' take the abs value of the difference between background and incoming and then threshold:
grayDiff.absDiff(grayBg, grayImage)

' find contours which are between the size of 20 pixels and 1/3 the w*h pixels.
' also, find holes is set to true so we will get interior contours as well....
contourFinder.findContours(grayDiff, 20, (340 * 240) / 3, 10, False) ' find holes
End If

If KeyHit(key_escape) Then
End If
End Method

Method draw()
Local numBlobs:Int
Local blobArea:Float
Local centerx:Float
Local centery:Float

' draw the incoming, the grayscale, the bg And the thresholded difference

SetColor $33, $33, $33
DrawRect 360,540,320,240

' we could draw the whole contour finder

' or, instead we can draw each blob individually,
' this is how to get access to them:
For Local blob:ofxCvBlob = EachIn contourFinder
numBlobs = numBlobs + 1
blob.draw(360, 540)
blobArea = blob.getArea()
blob.getCentroid(centerx, centery)


SetColor 255, 255, 255
DrawText numBlobs, 10, 10
'DrawText blobArea, 10, 20
DrawText centerx, 10, 30
DrawText centery, 10, 40
End Method

End Type


Armitage 1982(Posted 2011) [#53]
I lost track of that module !

Lately I wanted to learn a bit of processing but remember about something similar in BlitzMax.

What is the statut of ofx.mod ??
I try to build the fluid example but fail, most of the core examples still run correctly.

Last news on the official website is from 2009 :s
Maybe better to stick to real Processing ?

Ferret(Posted 2014) [#54]
The OpenCV examples fail.

format_codebox('Building haarfinder
flat assembler version 1.69.14 (963494 kilobytes memory)
3 passes, 4928 bytes.
Warning: .drectve `/DEFAULTLIB:"LIBC" /DEFAULTLIB:"OLDNAMES" ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:uuid.lib ' unrecognized
Warning: .drectve `-defaultlib:uuid.lib ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
Warning: .drectve `-defaultlib:LIBCMT ' unrecognized
Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized
D:/BlitzMax/bin/ld.exe: reopening D:/BlitzMax/mod/ofx.mod/ofxopencv.mod/examples/haarfinder.exe: Permission denied
D:/BlitzMax/bin/ld.exe: final link failed: Permission denied
Build Error: Failed to link D:/BlitzMax/mod/ofx.mod/ofxopencv.mod/examples/haarfinder.exe
Process complete