My initial concept was to experiment with a new emerging technology used for interaction and integrate it with an older generation game. In creating a new interface for an old, arguably outdated game style, this could spark new interest in the game and possibly inspire new uses for the new technology.
Aims, Considerations & Purpose
My main aim when starting this project was to come up with a highly interactive educational tool. I wanted to use new technology and apply it in an engaging way to encourage primarily children learning essential maths skills. I focused on key stages rather than ages, as I feel that such a tool could be just as useful for adult education as it is for children.
One of the considerations for this project was the resources and hardware needed. If this was to be developed into a classroom based tool, the price of equipment would need to be kept to a minimum. My interface requires a webcam (the one seen in my video cost £3.86 from Amazon), laminated sheets of paper which act as whiteboards and finally some dry-wipe markers.
The program interface aims to be bright and colourful to help make the learning experience fun. I created a character (Elfunzo) for users will follow along his quest. Initially this was not a focus, so still needs to be developed, as it would be useful to help maintain learners' interest and encourage them to return.
The program detects which worksheet is in use by the QR code. By holding any worksheet up to the webcam, the program will provide a problem relevant to that specific sheet. Each sheet has brief instructions which are is visible to the user, but the webcam will not detect them. All of the worksheets involve the user drawing within the black rectangle. The 'space' key is used to check any answers given.
The data returned by the QR code is a url with a four character prefix, eg: "fr01.motmit.co.uk". The program takes the four first characters to check which worksheet is being used. By using prefixes in front of a domain that I own means that if people scanned the QR code with their phones (not it's primary purpose) I could still direct them to a specific page of my choosing. The code also returns four position points as highlighted on this image.
By using the position vectors between the QR code position points returned, I worked out estimate points for the inner corners of the rectangle. By then comparing my estimate of where the rectangle contour should appear, with the actual contours detected I am able to find which contour in the image is the rectangle. The image here shows the inner rectangle contour being used (highlighted in green). As I developed this project I changed the code to use the outer rectangle contour. This then enabled the program to detect lines drawn from the very edge of the rectangle.
After detecting the rectangle contour, the program takes the four points of the image, cuts out the image and warps the perspective. This should fix any issues created by the user not holding the worksheet parallel to the camera. By doing this it also means that no matter what size of worksheet is detected on the camera it will always produce the same size 'cut out' image. This makes detecting the relative position of detected marks on the image much easier. The image here shows the still frame where a QR code has been detected and the output image directly underneath.
The program then runs contour detection on the output image. I have defined filters which filter out unwanted contours as well as unwanted points within each of these contours. I remove any contours which are too close to the edge of the screen or too close to a previously detected contour (representing a repeat detection). The image here shows an output image with multiple contours detected. Each point is highlighted by a red number. Even though I am not using it for my prototype at the moment this shows that detecting different shapes is already possible with my exsting code; it just needs parameters setup to define what to detect. At the moment in this image it is just treating every contour as a line so drawing a line between the minimum and maximum x points.
This image shows the output image being run through the code which has parameters set for lines and points. Lines are defined by contours which only have two points. Points are defined by contours with 5 points as well as at least 3 deleted points (due to repeat detection in the centre).