Monday, 31 October 2011

Edge Detection Implementation

When I was looking for examples of canny edge detection implementations in Java, I found code developed by Tom Gibara:

This implementation takes an image RGB, grayscale and performs a canny edge detection and returns a buffered image.

The useful code to perform the canny was extracted and implemented with the image produced from the background removal.  The output can then be compared with the result obtained from Matlab.

Thursday, 27 October 2011

Canny Edge Detector

A Gaussian filter is used to reduce the noise.  The image is convolved with the Gaussian filter


Next is to find the edge and determine the direction and gradient.  Four Sobel masks are used to find the edge gradient and the sum of all four is found and then the magnitude and angle can be found.


Using threshold value for the gradient and magnitude, the edges are filtered out


Then trace along the lines to give the canny image.

The code used in this section was created by Tom Gibara http://www.tomgibara.com/computer-vision/canny-edge-detector

Wednesday, 26 October 2011

Dynamic URL's

It is desired that the file is automatically uploaded to the server. One concept is to use a dynamic URL which would automatically populate the form on the page.

Dynamic URL's have the Form:

http://www.localhost:8080?value=object

value is the variable
object is its data value

The variable could then be extracted to the JSP using the code:

<% String Name = request.getParameter("value")%>

Initial image analysis

Once the image has been taken using the app, it is converted into a grayscale image using Matlab.  Peaks  with an intensity above 4000 between 54, 85 and 92, 149 where these values correspond to the grey colour value.  Removing these colour shows 2 out of the 5 balls.

Using each colour intensity RGB components, histograms for each pixel were made.  All of the green components were removed.  Using a maximum intensity of 25000, the red values cut off 67-133 and 67-135 gave better values with all 5 balls found but white patches on the table.

Changing the red and blue bands to give the best results, 5 balls found and edge of the table, the felt was blocked out.  If it finds a red between 85 and 151, a blue between 47 and 170, and green between 0 and 255, these values are set to zero and all other values set to 255 (white)

Matlab has a function 'edge' which can apply different edge detection techniques: canny, sobel, log, roberts, prewitt, and zero cross.

The best results were found using the canny technique.

Tuesday, 25 October 2011

Geometry Section - Physics Engine

The links on this page should be very useful for the pool engine

http://gamedev.stackexchange.com/questions/7862/is-there-an-algorithm-for-a-pool-game

File Upload Form

A form was made on the server so that the user could manually upload their photo to the server

Making a prototype

In order to ensure that we can make a basic program to complete the task, we decided that it is important to design and make a prototype.

For the Physics Engine section, a very simple program to calculate if the balls and cue are directly lined up with a pocket, using gradient calculations.  Another challenge will be to get the three distinct sections to transfer information between them.

The deadline for this prototype is next Wednesday (2nd November).

Grace

Monday, 24 October 2011

Server Requirements

The server must be able to do the following tasks:

  • Accept Image from the Mobile Device
  • Pass Image to Image Analysis which would receive Co-ordinates and Angles
  • Process approx 30 variables
  • Pass Values to the engine
  • Take Output values and display a hit or miss to user
Apache Tomcat Web Server was chosen, with pages running JSP's  http://tomcat.apache.org/

The image analysis and physics engine were to be written in Java


Stages in the Data Acquisition and Image Analysis


  1. Unwarp the image
  2. Background removal
  3. Ball detection
  4. Table Outline detection
    • rotation/ grid/ coordinates

Sunday, 23 October 2011

Possible Server Languages

Client Side Languages->Program is performed by the web browser using the users processing


Server Side->Program is Run on the server and output is displayed to the user, this is the preferred option

Server Side languages include:

  • Java via JSP pages
  • Javascript
  • PHP
  • Python

Tuesday, 18 October 2011

Using Matlab

Found Matlab function for colour detection.  Image file is opened and converted to RGB values.  These values are analysed and if they are between a given range, a 2D array for the given pixel is given an integer value.  Depending on the integer value, the dimensions of the ball can be calculated.

Another example Matlab function was found called Edge Detection.  This function finds the edges of coins and puts a green line around each coin.

Saturday, 15 October 2011

Physics Engine Requirements

The Physics Engine must complete the following tasks:
  • Receive the information passed from the image capture.
  • Calculate whether or not the ball which is being aimed at is going to be hit.
  • Calculate a potential shot in which the ball will be potted.
  • Calculate amount of correction needed in the angle of the cue to achieve potential shot.
  • Pass information to user interface
Ideally, the first task should be to find an existing model that can be adapted to fit the task requirements.

Friday, 14 October 2011

Android Development Tutorials

  http://www.youtube.com/user/thenewboston#grid/user/2F07DBCDCC01493A

This is a great page with 200 tutorials on developing for Android.
We've found this very useful, and really explains it for complete Beginners like us

Thursday, 13 October 2011

Considerations, Problems and Algorithm

Graphical User Interface: Problems and Deliverables

Deliverables:

The GUI needs to be functional in its operation in linking the user to the algorithms and calculations contained within the application. This needs to be simple and intuitive to use for a new user and appealing and easy pleasing to a regular user.

The GUI also acts as the controller for the other functions (Image Capture and Geometry) and calls them when the user requires them and uses info from them to tell the user if the shot is going to be a hit or miss.

 
The GUI will also take the information from the geometry, and passes it on to the user in the form of a ‘beep’. This ‘beep’ will be calibrated so that the gaps get smaller the closer the shot is to the pocket, until a constant ring when the shot will be a hit.



Expected Problems:

  Application runs too slowly for it to be useable, time will need to be spent improving the speed and reduce the size of the program.
  Bugs in the code prevent it from running; time will need to be spent testing program and all possibilities within it to weed out problems.



 Potential Problems:

  Information passed to Image capture is wrong or not detailed enough.
  Information from Geometry is wrong or incorrect.
  Design is not functional or eye catching
  Completion of code does not meet time schedule
  The GUI is functional, but when other sections are added it no longer is.

 
 Information to be passed to Image Capture:

  Turning on the camera and preparing it for image capture

Information to be received from Geometry:

  Whether the shot is going to be a hit or miss
  If the shot is a miss, the distance the shot is away from the pocket
 
Steps in the Design and Implementation of the GUI:

  Two Algorithms designed, one which starts with user and ends with image capture and another which starts with geometry and ends with ‘beep’ to user.
  A design drawn out and agreed upon
  Design created and coded, as an empty shell with no functions within
  Design approved
  Main function added i.e. Screen shows the image seen by the camera and the image capture function is called upon.
  Noise alert function added
  Noise alert function tested using self inputted numbers
  Noise alert implemented with numbers from Geometry function
  Testing application with full implementation of the image capture and geometry.

Optional Extras:

  Options menu for user to personalize the way the application is run or shown
  Recorded statistics for the users hit, miss ratio etc
  Use of accelerometer within the phone to make sure the phone is flat
  Use of web server to display the image seen by the camera on a computer screen, and line of the expected path plotted 
Algorithm for the GUI

  Image Capture and analysis: Problems and Deliverables

Deliverables:

The image capture and analysis needs to efficiently obtain an image using the mobile phones camera when held above the pool table. Once this image has been captured the analysis can be undertaken. From the image analysis the table’s dimensions such as the edges and pockets need to be defined. Also each ball needs to be located on the table and each ball differentiated from each other, the angle and direction of the cue need to be defined. All this information must be sent to the geometry calculation section so the shot assistant can be calculated.

Problems

    Not enough of the table is ‘in view’ using the camera so the shot cannot be predicted when the data is passed to the geometry calculation.
  Data from the image analysis could be in the wrong format for the geometry calculations
   None of the balls could be detected.
  The mobile phone is not level so that the image analysis creates incorrect locations for the balls and cue.
    Mobile phone is moved about to much so that an accurate image could not be taken
·  
Information to be sent to Geometry Calculations:

The geometry calculation sections will need the following information so that the calculations for the shot can be executed. They are;

  The table’s dimensions such as the boundary of the table and the location of each of the pockets.
  The different ball types such as the difference between the coloured and striped balls and the cue ball which is white.
  Ball location, the position at which each ball is on the table.
  The initial cue angle held by the player.

Information to be sent to the GUI:
  The new image needs to be displayed on the screen of the mobile phone.
  If the image being captured is not big enough for the full analysis I.e. the edges of the table of pockets could not be seen.


Information to be received from GUI:

  The GUI will prompt the image capture section when the camera needs to be turned on.
  Image capture algorithm will be prompted to be executed by the GUI.  
  The GUI will send the sounds. 

Steps in the Design and Implementation of the Image capture and analysis:

  Create a program which takes an image with the mobile phone’s camera ready for the analysis.
  Create and implement an algorithm for defining each the table dimensions.
  Choose an appropriate way of storing the information which will be sent for the geometry calculations.
  Create and implement an algorithm to distinguish the different types of balls and the balls location on the table.
  Create and implement and algorithm to detect the angle and direction of the cue in relation to a specific ball. 
  Send actual information the geometry calculations section

Algorithm for the Image Capture


Geometry Calculations: Problems and Deliverables

Deliverables:

Geometry Calculations section is required to take the information passed from the image capture and analysis and calculates whether or not the ball which is being aimed at is going to be hit. Once the ball has been targeted an algorithm needs to calculate a potential shot in which the ball will be potted and the amount of correction needed by changing the angle of the cue in relation to the cue ball.  

Expected Problems:

   Balls cannot be lined up with hole (no direct shot).
  Confused by 2 balls on table – aiming for wrong one but correct colour.
   Aiming for wrong user’s ball.


Potential Problems:

   Code runs too slowly to calculate correct cue position in the allocated time.
   Program calculates balls heading in wrong direction.

Information to be sent to the GUI:

    Whether the current shot is going to be a hit or a miss.
   The distance the cue needs to be moved to correct the shot.

Information to be received from image capture and analysis:

   The dimensions of the pool table such as the edges and the location of each pocket.
   The location of each ball and what type of ball they are.
   The angle of the cue with relation the cue ball.

Steps in the Design and Implementation of the geometry calculation:

   Create calculations from collision theory to predict which direction the balls will move in.
   Design an algorithm with for the collision movement.
   Create the program and test with ‘fake’ values.
   Test with real values obtained from the image capture and analysis. 
 •  Send the proposed improvements to the GUI.

Algorithm for the Geometry




Data Acquisition and Image Analysis Requirements


The following tasks are what is required:
  • Automated Data Acquisition
  • Automatic Detection of ball the positions
  • Automatic Detection of the table
  • Automatic Detection of the cue and its angle to the ball



Saturday, 1 October 2011

Learning the Basics

In order to learn the skills we will need for this project, we have been following tutorials on Youtube about creating android applications.


Within the project we will be using Eclipse and Android 2.2


We will learn about:

  • XML layouts
  • Threads
  • Using different screens
  • Using the camera
  • Buttons
  • Timers

Andy