OpenCV vision with EMC/linuxcnc

HI EMC forumites,

I'm really sticking my neck out here, but I'd like to start the ball rolling with the pick and place guys... I'm a vision systems designer and I'm familiar with openCV, I've got an engraving machine and use EMC. Combined with the fact I design and build a lot of high spec electronics, a pick and place machine seems a logical progression.

I've read as much as I can find on integrating vision into EMC, I'm keen to make a start but I'm not familiar enough with EMC's structure to get things going.

Is there anyone out there who is willing to collaborate on this work? Ideally any willing volunteers would have an in depth knowledge of the core of EMC. I'm happy to contribute the eventual works back to the EMC/linuxcnc community.

I hope we can make a really solid platform for intelligent vision systems in EMC, the pairing of OpenCV and EMC seems to good to overlook.

My email is This email address is being protected from spambots. You need JavaScript enabled to view it. - I'm keen to hear from anyone who's interested in helping out...

Best regards,
Ed

Can you explain how you see the control loop working?
Does the openCV just send a simple signal back to linuxcnc or complex movement commands?

cmorley wrote:
Does the openCV just send a simple signal back to linuxcnc or complex movement commands?

I imagine it would provide measured positions in a HAL component, for example fiducial line co-ordinates.
But I agree, the first part of the puzzle is to work out what the camera is for.

This is exactly why I thought I should publicly announce my intention of working on this - the ideas will hopefully be free flowing and we can quickly pick a sensible route.

I can write imaging routines that will find chosen objects in machine coordinates, i.e. fiducials, SMT components etc. I'd also propose functions for measuring rotation of parts for pick'n place.

I'm flexible, can implement this in any way that suits EMC and our needs - I'd just like some clear direction before I get too busy with it.

Building numerous small functions into a HAL component seems the best way to go - looking at the HAL layer, it's seems ample for my needs.

Thanks for looking...

Ed

I wish I could offer wisdom in the LinuxCNC area, but alas I am still much a newbie here.

I can offer first hand insight to how MY commercial PnP works. It seems the vision section provides offsets to the control for 2 things -

1) when a new board is loaded, the control sends the fudical camera over to "measure the board using the board fudicals. This in turn gets recorded as a global offset for the session. It easures both X and Y position, also theta or board rotation.

2) There is a second (and in my case a third for larger parts) camera that takes a picture of each component before it gets placed. This part of the system compares the part to a package definition for the part being placed and in turn seems to provide offset info simular to the 1st step; X,Y and Theta. As a secondary thing, it also confirms there is a part and the part is good. ie: if a pin is bent outside the package def it will trash the part and get a new one.

A good PnP should have 2 cameras to work as described above.

Now in a milling or lathe operation, Assuming we can achieve the same accuracy as my PnP, I could see a camera checking parts before maching takes place much like step 1 above. This would be a HUGE HUGE thing in the machiine industry. I can't tell you the number of parts I have had to scrap simply because an operator loaded it wrong and hit go.

Next could be a visual inspection function, like a poor mans CMM. Here the vision system would just provide coordinates of features that can be either compared to a referance or spit out an inspection sheet.

In a lathe, diameters and features could be checked and maybe re-cut if not up to snuff. Again here the camera would provide dimensions.

And last thing I can think of for now, a simple tool height gage/tool checker. Here the camera would be mounded in a fixed location and the tool presented to it. If its a new tool, the tool offset would be calculated. If it isan in use tool, it could be looked at to see if broken or worn.

So based on this long post - my opinion to the question of "what will the camera be for", is a vision system able to provide measurements/dimsions in what ever form LinuxCNC needs them. At a minimum this will be X,Y. But woud be GREAT if theta can also be done.

I am more than willing to help in any way I can to help with this goal. I think this could put LinuxCNC so far ahead of every thing else (not that it isn't already).

There ya go being all practical and what not

You mean since I can't even see through the doors, The inside mounted camera would be blind also? Hmmm thats a tough nut to crack. Now you got me thinking about scara mounted camera to move in when needed much like my touch probe does on my lathe..