HomeMate

#team Impresso
Ashish, Abinash and Piyush

HomeMate is complete solution for home automation. All it needs is a simple wearable device and the existing home appliances that need to be controlled. Sounds interesting! Right?

We aim to bridge gesture recognition technology and automation. User just needs to perform gestures in the direction of target device. Say, rotate wrist to increase intensity of light or lower the hand to slower speed of a fan. The solution involves getting input as gesture feed from the wearable device consisting of accelerometer, compass sensor, a detector circuit for hand gestures.
Features like degree of rotation of the fingers, accelerometer reading and pointing direction were used for prediction. After prediction, the corresponding output determines the function to be performed which are assigned for each gesture. Thus from speeding up fans to dimming lights, HomeMate is on work. And all this packed up in a very simple but intuitive user interface application.

Verse, Gesture to Speech

Verse, is wearable gadget to convert gestures to Speech.

Gestures are recorded from Potentiometer based wearable device with accelerometer and gyroscope sensors.

Data is transmitted to Android Device via bluetooth.

Android device predicts the gestures based on serial data received using kNN machine learning algorithm.

After prediction text is converted to speech.

In-Air Gesture Recognition using ANN trained data set

https://github.com/piyushbang/shape_recognizer/blob/master/shape_recognizer.cpp

Correct Classification rate : 88.88%!!  Best accuracy so far!!

Update: Working on OCR using current model with proper feature selection based on Teccaract OCR Model and other features for input to ANN. Stay tuned :).

Three main tasks:

1> Create dataset for training. Using Webcam, hand typed gestures are recorded. three gestures for now: Triangle, Rectangle and Circle.

2>Convert the images recorded into features. For starting sake, I resized the output images to 16*16 and took these 256 pixel values as features. Converted them into linear data to be used in for ANN.

3>Train the network using Training database and check the accuracy using test data set. Create an output param.xml file that store the ANN calculated feature table. This is later used for real time identification.

Task one: read database.

task two: Train Database.

task three: Predict the shape.

All these three functions are:

1>void Feature_set() and void read_dataset(char *filename, cv::Mat &data, cv::Mat &classes, int total_samples)

2>void train_data_set()

3>int predict_class(Mat image)

Git repository for the code is there for help. Any other queries are welcomed.

FishEye Uncurling

This video is of software by i2V Systems.

We during our internship developed this module.

Designed uncurling algorithm and implemented with proper user interface along with various perspective views.

All thanks to Sumit Aggarwal Sir, Director i2V Systems for his guidance.

Digital Pen/Typing

Git Repository/C++ file link:

https://github.com/piyushbang/Gesture-Typing/blob/master/gestureTyping/gestureTyping.cpp

In this project simple color selective background extraction is done.

Then the result is converted to binary.

Resulting image is pre-processed by dilating and thus removing errors in detection.

Contour is detected. We assume we have only one contour detected in image. If more than 1 is detected, image is cleared (text erased).

Then centre of contour is found and used to plot line between current centre and previous centre and thus a continuous writing is made possible.

Finger Counting using Image Processing

This project involve many steps to finally output the count of fingers, printed on screen.

I have use openCV 2.4.5 C++ libraries to implement the project.

Step1> Take 7 ROI where on your palm lies. Take average color in each of ROI. Median of each ROI is taken as required palm color.

Step2> Palm is extracted from background with appropriate threshold value.

Step3> Extracted image is converted to binary image and scaled down.

Step4> Contour are extracted.

Step5> From contour of maximum area, Convex hull and hull defects are identified.

Step6> This helps in calculation of finger tips. Each Defect indicate finger presence. Unwanted tips are removed using defect removal with threshold below minimum.

Step7>From finger tip data, counting is done.

Some mathematics are applied in case the count is 1. Since in this case too, hull defects are 0, same as in case of 0 fingers. So we check for highest point in hull and compare it with other contour points. If difference is large than others, (threshold value) we take it as 1 finger.

Step8> Result is printed on screen using puttext.

Gesture Controlled Vehicle

Task: To drive a robotic car with motion controlled by gestures and orientation of hand (sensor)

Devices and apparatus required:

• A Robot Chassis
• An accelerometer (I used MMA7660FC)
• Microcontroller board
• Arduino software/ RobotC

How to:

Concept is simple as the project. Accelerometer is a sensor that give analog signals according to the orientation of the sensor.

Sensor in used in active mode. (There are various mode like power saver mode etc.)

Wait! Sensor gives data or output using I2C protocol of communication. So IC is needed to write the address of the register you want to use and in this case we use active mode address 0X07. Use data sheet to know about other address of so many features of this accelerometer.

Then we just have to read the register which have in it the analog value corresponding to x axis and other for y axis.

Here we read 0X00 for x axis and 0X01 for y axis.

Value ranges for 0-65

Code is self-explanatory….

I did it in ROBOT C. Libraries of LCD and TWI/I2C need not to be provided here. But if anyone need it just let me know. Code’s explanation is quite clear!

lcd_init();                                                //Initializes LCD

lcd_gotoxy(0,0);

lcd_string(str0,13);

i2c_master_write(0x07,0x01);//Active Mode

wait1Msec(1000);

while(1)

{

lcd_clear();

/******************************** X – Data ***************************/

x=i2c_master_read(0x00);  //Read X-OUT Register

lcd_gotoxy(0,0);

lcd_string(str1,2);

lcd_showvalue(x);

/******************************** Y – Data ***************************/

y=i2c_master_read(0x01);   //Read Y-OUT Register

lcd_gotoxy(0,7);

lcd_string(str2,2);

lcd_showvalue(y);

/******************************** Computation ***************************/

if((x>12) & (x<25))

{

//Right Turn

SensorValue[M0] = false;

SensorValue[M1] = false;

SensorValue[M2] = true;

SensorValue[M3] = false;

lcd_gotoxy(1,0);

lcd_string(str3,5);

}

else if((x<55) & (x>45))

{

// left turn

SensorValue[M0] = true;

SensorValue[M1] = false;

SensorValue[M2] = false;

SensorValue[M3] = false;

lcd_gotoxy(1,0);

lcd_string(str4,4);

}

else if((y>5) & (y<25))

{

//Forward

SensorValue[M0] = true;

SensorValue[M1] = false;

SensorValue[M2] = true;

SensorValue[M3] = false;

lcd_gotoxy(1,0);

lcd_string(str5,7);

}

else if((y<55) & (y>45))

{

//Backward

SensorValue[M0] = false;

SensorValue[M1] = true;

SensorValue[M2] = false;

SensorValue[M3] = true;

lcd_gotoxy(1,0);

lcd_string(str6,8);

}

else

{

//Stop

SensorValue[M0] = false;

SensorValue[M1] = false;

SensorValue[M2] = false;

SensorValue[M3] = false;

lcd_gotoxy(1,0);

lcd_string(str7,4);

}

wait1Msec(100);

}

My first Robot!

Micro Controller Board

Any microcontroller board is a PCB that have connections made to the pins of a microcontroller chip. A microcontroller chip is ready to go if it has a power supply, a ground connection, an oscillator and a reset pin.

So before making connections of uC pins on a board, we should know what different pins are meant for.

                                                              ***************************

Pin Descriptions (*from data sheet)

Pin 10- VCC Digital supply voltage.

Pin 32- GND Ground.

Port A (PA7-PA0) Port A serves as the analog inputs to the A/D Converter.

Port A also serves as an 8-bit bi-directional I/O port, if the A/D Converter is not used. Port pins can provide internal pull-up resistors (selected for each bit). The Port A output buffers have symmetrical drive characteristics with both high sink and source capability. When pins PA0 to PA7 are used as inputs and are externally pulled low, they will source current if the internal pull-up resistors are activated.

The Port A pins are tri-stated when a reset condition becomes active, even if the clock is not running.

Port B (PB7-PB0) Port B is an 8-bit bi-directional I/O port with internal pull-up resistors (selected for each bit). The Port B output buffers have symmetrical drive characteristics with both high sink and source capability. As inputs, Port B pins that are externally pulled low will source current if the pull-up resistors are activated. The Port B pins are tri-stated when a reset condition becomes active, even if the clock is not running. Port B also serves the functions of various special features of the ATmega16.

Port C (PC7-PC0) Port C is an 8-bit bi-directional I/O port with internal pull-up resistors (selected for each bit). The Port C output buffers have symmetrical drive characteristics with both high sink and source capability. As inputs, Port C pins that are externally pulled low will source current if the pull-up resistors are activated. The Port C pins are tri-stated when a reset condition becomes active, even if the clock is not running. If the JTAG interface is enabled, the pull-up resistors on pins PC5(TDI),  C3(TMS) and PC2(TCK) will be activated even if a reset occurs.

Port C also serves the functions of the JTAG interface and other special features of the ATmega16.

Port D (PD7-PD0) Port D is an 8-bit bi-directional I/O port with internal pull-up resistors (selected for each bit). The Port D output buffers have symmetrical drive characteristics with both high sink and source capability. As inputs, Port D pins that are externally pulled low will source current if the pull-up resistors are activated. The Port D pins are tri-stated when a reset condition becomes active, even if the clock is not running. Port D also serves the functions of various special features of the ATmega16.

RESET – Reset Input. A low level on this pin for longer than the minimum pulse length will generate a reset, even if the clock is not running. Shorter pulses are not guaranteed to generate a reset.

XTAL1 Input to the inverting Oscillator amplifier and input to the internal clock operating circuit. XTAL2 Output from the inverting Oscillator amplifier.

AVCC – AVCC is the supply voltage pin for Port A and the A/D Converter. It should be externally connected to VCC, even if the ADC is not used. If the ADC is used, it should be connected to V through a low-pass filter.

AREF – AREF is the analog reference pin for the A/D Converter.

*********************************

So in brief all pin have some of other function attached to it other than being simply Input or output. This is specifically the register attached to that pin which when activated or set HIGH or ON by writing on particular registers, performs the particular task.

Oscillator is connected to XTAL1 and XTAL2.

Ground is connected to pin 32 and VCC to pin 10.

AVCC is shorted to VCC.

AREF is also given 5V supply.

RESET is active LOW pin. If a 0 volt signal is given when input power is also HIGH, the code in uC restarts.

We also want that at every time we start our uC, it starts with a reset. So a capacitor is connected to RESET pin. So when power is supplied to board, uC is ON but reset is LOW as it takes some to time to charge and reach 5V level. So it RESET’s on every start. A switch is also connected that is grounded and connected to RESET, so when it is pressed, capacitor discharges and it RESET’s.

Schematic for a simple microcontroller board is here below:

And its PCB layout:

Above circuit connects Port A, B, C and D with a port pattern.

PORT A further have LED’s and switches connected via resistors.

Another important circuit is the Programming Port! It is port having MISO, MOSI, SCK, RESET, VCC and Ground. Through this a programmer is connected via which code is written from computer to uC made possible by communication set between the uC and PC through another uC in Programmer.

Oscillator pins are severely affected by noise signals so they are kept as near to uC as possible. Two capacitors are connected to input and output of oscillator and other end is grounded. Note that the circuit for oscillator is already present in uC but due to fabrication limitations we need to connect to it externally.

Power port is also important to connect peripherals to boards and power and ground them.

To end the blog I would mention about the Arduino board and similar boards. These are ready made platform and are very reliable as connections are precise. The board do not require a programmer to write a code in it. For this purpose a part of memory of uC is dedicated and Bootloader is programmed to run every time you write a code.

This makes it free from external uC to help it communicate with the computer.

Thanks for reading!

 

 

 

 

Introduction

Protocol32 is a platform to share my knowledge of robotic and learn while I write the blog. It shall have varied topics related to robotics and shall dwell mainly to make projects and in the process learn what all it needs to make it.

To begin with we should know what robotics is?

No formal definition here!

It’s a broad field. To make a technology that simplifies your task, say a device that locate where you placed your T.V. remote, or that connects the real to the virtual, say a system that recognizes your gestures to get things done. In brief, knowledge of Robotics empowers you to make the technology yourself.

Mechanics is a major part of the system that we wish to make. Designing it properly simplifies the task and make it function efficiently. But for now we shall focus mainly on how to make it work autonomously.

Autonomous robotics: Mechanical system that is guided by an embedded system.

Embedded system may be a microprocessor or a microcontroller.

To state in simpler words, Signal from external environment, say an I.R. sensor, is received and processed as input on I/O pin and then interpreted to give output on its I/O pins which is again given to system to drive the actuator or motor.

A microcontroller is such an embedded system generally used. A microcontroller is chip that act as a brain. It has many pins. Some are configured input and others as output. These pins accept 5V supply as input and also give 5V as output.

There are many other functions of pins of uC (microcontroller) apart from using them as input or output such as they can be used as Interrupt input pins, PWM output pins, ADC pins etc. and are only used to communicate with other board and computer using UART/SPI/I2C/CAN etc.

We shall learn about other functions ones we are done with I/O pins.

So to start working on an autonomous robot we must have following:

• Microcontroller
• A microcontroller board (or we can make it ourselves!)
• A programmer – A set of wires and a pre-programmed uC, to connect our uC to the computer.
• Some LED’s
• A software/ environment to code our microcontroller – say Eclipse for AVR, Eclipse for ARM, Arduino, these convert or compile our code into Assembly line language. We can directly write our code in assembly line and write it on our uC.

 

Arduino is a very awesome platform for doing robotics project. It has its own software for writing code on an Arduino board, and it has predefined set of libraries and functions that can use any pins of the uC board in the way we want.

Note: For beginners, if your aim is to make project and have no intension of using other boards like PIC or TI, go for Arduino. But if really know the concepts of uC and its basics, go for AVR. It will help you in long run and enable you to switch to any board in future easily.

This blog was an introduction to autonomous robotics. Next I shall write about what microcontroller board is and how it is made!

Thanks for reading!