Difference between revisions of "Product Evaluation (Sensors)"

From EG1004 Lab Manual
Jump to: navigation, search
(1003 --> 1004)
 
(109 intermediate revisions by 7 users not shown)
Line 1: Line 1:
= 3D Modeling and CAD Video =
General Engineering has created a video that covers the fundamental concepts of 3D modeling for the semester-long design project: [https://stream.nyu.edu/media/EG+1003+Lab+3A+3D+Modeling+and+Design/1_sgq36ht8/34608091 3D Modeling and CAD Video]. This video provides the fundamental principles behind the engineering design process and will help determine what solutions justify creating a 3D model. Material covered in the video will be on the lab 3 quiz. Viewing the video is a mandatory pre-lab assignment.
= Objectives =
= Objectives =
A specified robot to evaluate has already been created for this lab using an EV3 robotics kit. The design will be evaluated for accuracy and precision by utilizing two different sets of code, the first without the use of sensors and the second with. The robot will be tested on one of the two provided mini-courses. The data from the product evaluation of both sets of code will be used to determine if the robot should or shouldn’t use sensors.  
A pre-built robot will be evaluated for accuracy and precision by utilizing two different sets of code, the first without the use of sensors and the second with sensors. The robot will be tested on one of the three provided mini-courses. The data from the product evaluation of both sets of code will determine if the robot should or should not use sensors in navigating the mini-course.


== Product Evaluation ==
== Product Evaluation ==
Product evaluation is a common task for an engineer and for companies. Sometimes the product evaluated will be a company's product, and sometimes companies will be evaluating a competitor's design. In either case, this evaluation will use a '''''standard''''' (an accepted value), determine an '''''average''''', and calculate '''''accuracy''''' and '''''precision'''''.<ref name="one">Oakes, W.C., L.L. Leone, and C.G. Gunn, ''Engineering Your Future'', MI: Great Lakes Press, 2002.</ref>
Product evaluation is a common task for an engineer and for companies. Sometimes the product evaluated will be a company's product, and sometimes companies will be evaluating a competitor's design. In either case, this evaluation will use a standard (an accepted value), determine an '''average''', and calculate '''accuracy''' and '''precision'''. <ref name="one">Oakes, W.C., L.L. Leone, and C.G. Gunn, ''Engineering Your Future'', MI: Great Lakes Press, 2002.</ref>


Professional engineers must be sure of standards. For the purpose of this lab, the EG1003 Standard will be 80% precision and accuracy. This means that the product has to achieve a percent accuracy and percent precision of 80 percent and it has to pass 80 percent of the tests that it undergoes.<ref name="one"></ref>
Professional engineers must be sure of standards. For the purpose of this lab, the EG1004 standard will be 80% precision and accuracy. This means that the product has to achieve a percent accuracy of 80% and meet a tolerance of <math> \pm </math> 1cm for distance and <math> \pm </math> 10 degrees for angle. <ref name="one"></ref>


When taking measurements, the first step is to determine an average.
When taking measurements, the first step is to determine an average (Equation 1).


<math>Average\left(A_p\right) = \frac{P_1+P_2+P_3+...+P_n}{N}\,</math>
<math>Average\left(A_p\right) = \frac{P_1+P_2+P_3+...+P_n}{N}\,</math>


In this equation, ''A<sub>p</sub>'' is the average, ''P<sub>1</sub>'', ''P<sub>2</sub>'', ''P<sub>3</sub>'', ''...'', ''P<sub>n</sub>'' are the results of the tests and ''N'' is the total number of tests performed.
Equation 1: Average.
 
In this equation, ''A<sub>p</sub>'' is the average, ''P<sub>1</sub>'', ''P<sub>2</sub>'', ''P<sub>3</sub>'', ''...'', ''P<sub>n</sub>'' are the results of the tests, and ''N'' is the number of tests performed.


The '''''average''''' is compared to the '''''standard'''''. Accuracy is a variable the must always be considered. The degree of accuracy required will allow the results to be verified. Engineers often work on projects that affect people's safety; it is critical that the solutions to the problems solved by technical professionals are '''''accurate'''''.
The '''''average''''' is compared to the '''''standard'''''. Accuracy is a variable the must always be considered. The degree of accuracy required will allow the results to be verified. Engineers often work on projects that affect people's safety so it is critical that the solutions to the problems solved by technical professionals are '''''accurate'''''.


The comparison of the average to the standard is called accuracy, defined as ''Acc'' in the following equation:
The comparison of the average to the standard is called accuracy, defined as ''Acc'' in Equation 2.


<math>Acc = \left|P_s - A_p\right|\,</math>
<math>Acc = \left|P_s - A_p\right|\,</math>


In this equation, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured. Accuracy is always a positive number, so use the absolute value in case ''P<sub>s</sub>'' is less than ''A<sub>p</sub>''.
Equation 2: Accuracy.
 
In Equation 2, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured. Accuracy is always a positive number, so use the absolute value in case ''P<sub>s</sub>'' is less than ''A<sub>p</sub>''.


Precision is the repeatability of a result, or how close the results are to each other, defined as ''Prec'' in the following equation:
Precision is the repeatability of a result, or how close the results are to each other, defined as ''Prec'' in Equation 3.


<math>Prec = \left|P_{high} - P_{low}\right|\,</math>
<math>Prec = \left|P_{high} - P_{low}\right|\,</math>


In this equation, ''Prec'' is precision, ''P<sub>high</sub>'' is the highest data value and ''P<sub>low</sub>'' is the lowest data value. Like accuracy, precision is always a positive number, so use an absolute value here as well.
Equation 3: Precision.


A product test can be accurate but imprecise, or precise but inaccurate (Figure 1). The goal is always the combination of accuracy and precision. In order for engineers to make a recommendation based on experimental test data, it is important to determine how a product compares to a standard. Engineers need these benchmarks to determine the quality of a product.
In Equation 3, ''Prec'' is precision, ''P<sub>high</sub>'' is the highest data value and ''P<sub>low</sub>'' is the lowest data value. Like accuracy, precision is always a positive number, so use an absolute value here as well.
 
A product can be accurate but imprecise, or precise but inaccurate (Figure 1). The goal is to achieve a combination of the highest accuracy and precision. In order for engineers to make a recommendation based on experimental test data, it is important to determine how a product compares to a standard. Engineers need these benchmarks to determine the quality of a product.


[[Image:Lab hardsyn 12.gif|frame|center|Figure 1: Accuracy and precision.]]
[[Image:Lab hardsyn 12.gif|frame|center|Figure 1: Accuracy and precision.]]
Line 38: Line 41:
<math>%Acc = \left ( 1 - \frac{ \left | P_s - A_p \right | }{P_s} \right ) \times 100%\,</math>
<math>%Acc = \left ( 1 - \frac{ \left | P_s - A_p \right | }{P_s} \right ) \times 100%\,</math>


In this equation, ''%Acc'' is the percent accuracy, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured.
Equation 4: Percent accuracy.


Percent accuracy shows how precise a product is compared to the standard. The equation for percent precision is:
In Equation 4, ''%Acc'' is the percent accuracy, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured.


<math>%Prec = 100e^{-BP}\,</math>
Precision shows how precise a product is compared to the standard.


In this equation, ''%Prec'' is the percent precision, ''B'' is the exponential decay factor, and ''P'' is the precision measured. Each individual test requires the use of a different exponential decay factor (''B''):
Adequate precision is determined by an acceptable tolerance, set by the manufacturer. In this lab, the precision of the robot must be compared to the tolerance allowed:
: Distance Test: ''B'' = 0.0972
: Distance Test: <math>T = \pm 1 cm</math>
: Angle of Deviation Test: ''B'' = 0.0323
: Angle of Deviation Test: <math>T = \pm 10 degrees</math>


Calculating percent accuracy and percent precision and then comparing those results to a standard determines the quality of the product.
Calculating percent accuracy and precision and then comparing those results to a standard determines the quality of the product.


== Quality Improvement ==
== Quality Improvement ==
Quality improvement is a process of analyzing a design and testing it either through physical modeling, computer simulation, or mathematical modeling. Quality improvement ensures that a product will perform as expected and allows improvements to be made to the design.
Quality improvement is a process of analyzing a design and testing it through physical modeling, computer simulation, or mathematical modeling. Quality improvement ensures that a product will perform as expected and allows improvements to be made to the design.


When deciding how to improve the design in this lab, keep these questions in mind:
When deciding how to improve the design in this lab, keep these questions in mind:
Line 59: Line 62:


Changes will be implemented after testing the initial code.
Changes will be implemented after testing the initial code.
== Reverse Engineering ==
Reverse engineering software involves reversing a program's machine code to obtain the original source code or determine the file structures the program uses. This needs to be done when source code is lost or is not available. For example, when a project is reactivated after being dormant for years to add new features to a product, the original design documentation may have disappeared. Also, when companies are writing a program that must interface to a second company's software, the second company may not have any incentive to cooperate by describing how the data in the files they use are organized, forcing the first company to deduce it on their own. Sometimes, this process is also under­taken as a way to improve the performance of a program, to fix a bug, or to find a virus. When the source code is obtained in this way for any of these reasons it is legal and necessary. Reverse engineering software in order to copy it constitutes a copy­right violation and is illegal.<ref name="two">''What Is ''website. TechTarget Network. Retrieved July 29th, 2003.</ref>
Hardware reverse engineering involves taking apart a device to see how it works. If a processor manufacturer wants to see how a competitor's processor works, the company can purchase the processor, disassemble it, and then make a new processor similar to it. In some countries, this process is illegal. Hardware reverse engineering is quite expensive and requires an expert in the field.<ref name="two"></ref>
= Design Considerations =
Carefully consider the desired results and what needs to be done to achieve it:
* What made or will make the results inconsistent?
* If the robot failed a test, why did it fail?


= EV3 Sensors =
= EV3 Sensors =
== Sensor Basics ==
== Sensor Basics ==
A sensor is something that measures some specific value and is used as an input. Different features can be measured by the EV3 sensors, such as light intensity, angle, or distance. The three sensors used in this lab are the light sensor, gyro sensor, and ultrasonic sensor.
A sensor is a device that measures a specific value and is used as an input. Different properties can be measured by the EV3 sensors, such as light intensity, angle, or distance. The three sensors used in this lab are the light sensor, gyro sensor, and an ultrasonic sensor.
=== Light Sensor ===
=== Light Sensor ===
The light sensor (or “color sensor”) can measure reflected light intensity, color, and ambient light, and is useful for when the robot has the option to follow a line or use differences in color to perform different actions (such as stopping when reaching a different color). If measuring color, the sensor outputs a value (0-7) that corresponds to a specific color. Measuring reflected light outputs a value (0-100) depending on the intensity of the light that is reflected. The ambient light works the same way as the reflected light, except it just measures the surrounding light. The light sensor, as well as all other sensors, can be found in the yellow category of MINDSTORMS:
The light sensor or color sensor (Figure 6) can measure reflected light intensity, color, and ambient light, and is useful when the robot has the option to follow a line or use differences in color to perform different actions, such as stopping when detecting a different color. If measuring color, the sensor outputs a value (0-7) that corresponds to a specific color. Measuring reflected light outputs a value (0-100) depending on the intensity of the light that is reflected. The ambient light works the same way as the reflected light, except it measures only the surrounding light. The light sensor, as well as all other sensors, can be found in the yellow category the Mindstorms software. The light sensor should be placed so that it is 2 cm, at max, away from whatever it trying to record.


[[Image:Light_sensor.PNG|thumb|center|640px]]
[[Image:Light_sensor.PNG|thumb|center|640px|Figure 6: Light sensor.]]


The type of value the sensor can measure can be changed by clicking the small box on the bottom left of the block:
The type of value the sensor can measure can be changed by clicking the small box on the bottom left of the block (Figure 7).


[[Image:Light_sensor_2.PNG|thumb|center|640px]]
[[Image:Light_sensor_2.PNG|thumb|center|640px|Figure 7: Light sensor, value input.]]


=== Gyro Sensor ===
=== Gyro Sensor ===
The gyro sensor can measure the exact degree it turns or the rate at which it is turning. For example, if the robot is set to turn 90 degrees using the gyro, the robot will keep trying to turn until 90 degrees is reached, regardless of any bumps in its way:
The gyro sensor can measure the degrees it turns or the rate at which it is turning. For example, if the robot is set to turn 90º using the gyro, the robot will keep trying to turn until 90º  is reached, regardless of any bumps in its way (Figure 8). The gyro sensor should be placed parallel to the plane of the EV3 Brick.


[[Image:Gyro_sensor.PNG|thumb|center|640px]]
[[Image:Gyro_sensor.PNG|thumb|center|640px|Figure 8: Gyro sensor.]]




=== Ultrasonic Sensor ===
=== Ultrasonic Sensor ===
The ultrasonic sensor measures the distance between the robot and the closest object to it. It can measure distance in both centimeters and inches; the recommended range is 1 cm to 250 cm:
The ultrasonic sensor measures the distance between the robot and the closest object to it. It can measure distance in both centimeters and inches; the recommended range is 1 cm to 250 cm (Figure 9).


[[Image:Ultrasonic.PNG|thumb|center|640px]]
[[Image:Ultrasonic.PNG|thumb|center|640px|Figure 9: Ultrasonic sensor.]]


== Switches and Loops ==
== Switches and Loops ==
Switches and loops are used to tell the robot what actions to perform depending on the inputs taken from its surroundings, measured by the sensors.
Switches and loops are used to tell the robot what actions to perform depending on the inputs from its surroundings, measured by the sensors.


=== Switch ===
=== Switch ===
A switch serves as an “if-else” statement: if the initial condition is true, the top action will occur, if false, the bottom action will occur. This block is used when the robot has to perform different actions depending on the situation it is in:
A switch is an “if-else” statement: if the initial condition is true, the top action will occur, if false, the bottom action will occur. This block is used when the robot has to perform different actions depending on the input from the sensors (Figure 10).


[[Image:Switch_example.PNG|thumb|center|640px]]
[[Image:Switch_example.PNG|thumb|center|640px|Figure 10: Switch.]]


=== Loop ===
=== Loop ===
A switch will only run the code once, so if repeated action is desired, a loop must be used to repeat the switch’s statement. A loop continuously runs the code inside of it until the end statement occurs. The end statement can be determined with a sensor, or hard-coded to a specific amount of time or rotations; the figure below shows an infinite loop:
A switch will only run the code once, so if repeated action is desired, a loop must be used to repeat the switch’s statement. A loop continuously runs the code until the end statement occurs. The end statement can be determined with a sensor or hard-coded to a specific duration or rotations. Figure 11 shows an infinite loop.


[[Image:Loop.PNG|thumb|center|640px]]
[[Image:Loop.PNG|thumb|center|640px|Figure 11: Loop.]]


= Materials and Equipment =
= Materials and Equipment =


* Mindstorms kit with pre-made robot
* A Mindstorms kit with pre-made robot
* Computer with Mindstorms software
* A computer with Mindstorms software
* Mini-course with accuracy markings
* A mini-course with accuracy markings


= Procedure =
= Procedure =


== Problem Statement ==
== Problem Statement ==
Evaluate two types of code for a robot by creating a line of code without the use of sensors and then a separate line of code using sensors. For each trial, record distance from the wall and angle of deviation. Test the first code by running four trials per section of the mini-course as marked on the physical course, giving you a total of eight trials for the first code. Create a second code using switches and loops to incorporate the types of sensors found on the robot and features on the mini-course. Test again with four trials per section. The total amount of trials for the whole procedure is 16. Download the pre-made [[Media:EG_Lab_3_data.xlsx|Datasheet]] and fill in the data for each trial.
Evaluate two types of code for a robot by creating a line of code without the use of sensors, and then a separate line of code using sensors. For each trial, record the distance from the wall and angle of deviation from the center line. Test the first code by running four trials of Section 1 of the mini-course as marked on the physical course. Create a second line of code using switches and loops that incorporate the types of sensors found on the robot and features on the mini-course. Test the robot again with four trials of Section 1. This part of the procedure will have a total of 8 trials. Download the pre-made [[Media:EG_Lab_3_data.xlsx|Datasheet]] and fill in the data for each trial.


== Course Breakdown ==
== Course Breakdown ==
The mini-course is divided into two sections: one flat straight section of four tiles and a second section consisting of a right turn and ramp. Both sections have a wall at the end. In the following schematic, Section 1 is marked in red and Section 2 is marked in blue:
The mini-course (Figure 12) has a designated Section 1 to complete the eight trials of the lab: one flat straight section of four tiles and a right turn with a wall at the end. The competitive trial will utilize the entire course. In Figure 8, Section 1 is marked in red.
[[Image:Course_diagram.PNG|thumb|center|400px]]
[[Image:Course_diagram.PNG|thumb|center|400px|Figure 12: Mini-course.]]


== Part 1: Hardcoding ==
== Part 1: Hardcoding ==
=== Section 1: Red ===
Based off of your observations, make design changes to the robot in order to run trials on the first section (Section 1) of the mini-course.
# Start the '''Mindstorms EV3''' software
# Click on the '''+''' in the top left corner to open up a blank program.
#: [[Image:Lab evalimprov ev3 2.png|thumb|center|500px]]
# The screen should look like this. Your program begins with the '''Start''' block.
#: [[Image:Lab evalimprov ev3 3.png|thumb|center|500px]]
# Place a '''Steering''' block from the '''Action''' palette next to the '''Start''' block. Set the '''Port''' to reflect both motors plugged into the brick; both ports have to be correct.
#: [[Image:Steering_block.PNG|thumb|center|640px]]
#: Along the bottom of the '''Steering''' block, there are four options. On the left, there is a Mode Selector button, followed by three values for the input parameters picture above the value: from left to right, Power, Rotations, and Brake at End.
#: [[Image:Steering_options.PNG|thumb|center|640px]]
# Change the type of the '''Steering''' block to '''On for Seconds'''. This will change the '''Rotations''' input parameter to '''Seconds'''.
#: [[Image:Steering_seconds.PNG|thumb|center|640px]]
# Test different values of '''Power''' and '''Seconds''' to find the best settings to travel the first four tiles of the course and stop '''5 cm''' from the wall. When the optimal settings are found, alert the TA and run the program. Run the program for a total of 4 trials. For each trial, record the distance between the robot and wall when the robot stops and the angle of deviation from the center line. Record the angle as though the straight line is 360 degrees.


=== Section 2: Blue ===
'''Every run, unless the robot goes off the course, counts as a trial. There is only one test run allowed after which all runs will be recorded as trials.'''
# Create the second half of the program to turn the robot and continue up the ramp, stopping '''10 cm'''. Test different values of '''Power''' and '''Seconds/Rotations''' to find the best settings to travel the last 5 tiles of the course. A sample turning code is provided below:
'''When troubleshooting the code, please cross check the port number on the code and the port the sensor/motor is connected to before anything else.'''
#: [[Image: Turn_hardcode.PNG|thumb|center|640px]]
=== Product Evaluation ===
# Run the program and make adjustments to the program. Once the accurate settings are found, perform 4 trials for this section, to bring the total count to 8 trials. Again, take distance and angle measurements for each trial.
Using the modified design, conduct four trials of the robot using a code without sensors to travel in a straight line across the first four tiles, turn 90 degrees to the right, and stop 5 cm from the wall. '''The values used in the sections of code below are only examples - the actual values must be determined through experimentation.'''
# Start the '''Mindstorms EV3''' software.
# Download the [[Media:Lab_3_Product_Evaluation.zip|zipped folder]] containing the Mindstorms program.
# The hardcoding program should look like the one pictured below: [[Image:Hardcode_2018.png|thumb|center|700px|Figure 17: Part 1 Code]]
# Please read the comments before running the program.
# Change the ports to the configuration on your robot.
# Run a sample trial to understand the code.
# Evaluate the number of rotations by measuring the total distance that the robot has to move for a certain block of code and divide that by the perimeter of the wheel.
# Change the values to the evaluated numbers.
# Try to stop 5cm before the wall.
# Run the program for four trials. For each trial, record the distance and angle using pre-made markings on the sides of the tile.
# Distance is measured as the distance from the wall for which the ruler on the side of the tile can be used. (Denote it as y)
# Angle of deviation is calculated in the following way:
## Measure deviation from the center of the wall. (Denote it as x)
## tan^-1 (x/y)
## '''If you are confused as to how to calculate the angle of deviation, ask your TA for help.'''


== Part 2: Sensors ==
== Part 2: Sensors ==
=== Section 1: Red ===
'''Every run, unless the robot goes off the course, counts as a trial. There is only one test run allowed after which all runs will be recorded as trials.'''
The first section of the course requires following a straight line. The following steps outline how to build a line-following code for the robot to stay on track.  
'''When troubleshooting the code, please cross check the port number on the code and the port the sensor/motor is connected to before anything else.'''
# Click on the ‘’’+’’’ in the top left corner to open up a blank program.
=== Quality Improvement ===
# Insert a blank '''Loop'''. Insert a '''Switch''' within the loop.
Using the results of the original robot, incorporate the use of sensors in the robot's code. The following steps outline how to build a line-following code for the robot to stay on track. '''The values used in the sections of code below are only examples; the actual values must be determined through experimentation.'''
#: [[Image: Loop_switch.PNG|thumb|center|640px]]
# Move to the Software Trials tab
# Set the input of the switch to a '''Light Sensor''' and the type to '''Reflected Light Intensity'''. Set the comparison to the “greater than” symbol and use '''Port View''' on the EV3 brick to measure the light intensity of the line. Set the value of the light intensity to the value displayed by Port View.
# The program should look like this: [[Image:Sensors_Code_2018.png|thumb|center|800px|Figure 18: Software Trials Code]]
#: [[Image: TAKE PIC OF PORT VIEW AND SHOW LIGHT INTENSITY SQUARE|thumb|center|640px]]
# Please read the comments before running the program
# In the top half of the switch, insert two '''Move Motor''' blocks. Set one to stop and one to continuously moving. Duplicate this in the bottom half but switch the ports. This way one wheel will move while the other stays stationary, which will make the robot wiggle along the line:
# Change the ports to the configuration on your robot
#: [[Image:Lab_sensorsev3_26.png|thumb|center|640px]]
# Run a sample trial to understand the code
# Use the '''Ultrasonic Sensor''' to make the robot stop 5 cm away from the first wall. Set the end statement of the loop to be controlled by the ultrasonic sensor and set the type to '''Distance Centimeters''' for 5cm.
# Using the port view shown below, set the readings on the Mindstorms code for the light sensors and the gyro sensors blocks to the appropriate values. Adjust Rotations as done in the previous part. '''Ask a TA for assistance if you don't know what values to use.''' [[Image: Port_view.PNG|thumb|center|640px|Figure 19: Port view]]
#: [[Image:Loop_options.PNG|thumb|center|500px]]
# Take the same measurements as for Part 1: Harcoding
 
# Tabulate results neatly in the Datasheet
=== Section 2: Blue ===
The second section of the course requires turning to the right 90 degrees, advancing forward one tile, and turning another 90 degrees to prepare for going up the ramp. Each time the '''Gyro Sensor''' is used, it has to be reset in order to set the direction the robot is facing before turning as zero.
# Insert a '''Gyro Sensor''' block and set it to reset:
#: [[Image:Gyro_sensor_reset.PNG|thumb|center|640px]]
# Insert a '''Wait''' block right after the gyro reset. Set it to 2 seconds.
#: [[Image:Gyro_wait.PNG|thumb|center|640px]]
# In order for the robot to continue turning until 90 degrees is reached, a '''Loop''' must be used to constantly run the robot until the gyro sensor reads 90 degrees. Insert a loop after the wait block
#: [[Image:Gyro_code_1.PNG|thumb|center|640px]]
# Insert a '''Steering''' block into the loop. Select the port that when activated will turn the robot to the right.
#: [[Image:Gyro_sensor_2.PNG|thumb|center|640px]]
# After the loop, insert a '''Move Motor''' block for the robot to travel straight one tile. Use either rotations or time to move the robot and duplicate the first part of the program after the motor block.
#: [[Image:Gyro_code_complete.PNG|thumb|center|640px]]
# Recreate the line following code to allow the robot to travel up the ramp. Test different values of power to overcome the ramp. The end statement should cause the robot to stop 10cm away from the wall.
#: [[Image:Section_2_final.PNG|thumb|center|640px]]


== Tabulation of Results ==
== Tabulation of Results ==
All raw data must be inputted into the [[Media:EG_Lab_3_data.xlsx|Datasheet]]. From the datasheet, determine if the EG standards were met for accuracy and precision:
All data must be inputted into the Datasheet. From the datasheet, determine if the EG standards were met for accuracy and precision; recorded precision must be divided by 2 to get the corresponding tolerance:
{| class="wikitable" style="text-align: center; margin: 0px auto;"
{| class="wikitable" style="text-align: center; margin: 0px auto;"
! Hardcoding !! Accuracy !! ''%Acc'' !! Pass/Fail !! Precision !! ''%Prec'' !! Pass/Fail
! Hardcode !! Accuracy !! ''%Acc'' !! Pass/Fail !! Precision !! ''Tolerance'' !! Pass/Fail
|-
|-
! Distance Test || || || || || ||
! Distance Test || || || || || ||
Line 179: Line 162:
|}
|}


Retest with new design using the same table, record the data.


{| class="wikitable" style="text-align: center; margin: 0px auto;"
{| class="wikitable" style="text-align: center; margin: 0px auto;"
! Sensors Coding !! Accuracy !! ''%Acc'' !! Pass/Fail !! Precision !! ''%Prec'' !! Pass/Fail
! Sensors Code!! Accuracy !! ''%Acc'' !! Pass/Fail !! Precision !! ''Tolerance'' !! Pass/Fail
|-
|-
! Distance Test || || || || || ||
! Distance Test || || || || || ||
Line 188: Line 170:
! Angle of Deviation Test || || || || || ||
! Angle of Deviation Test || || || || || ||
|}
|}
== Part 3: Robot Competition ==
The robot competition consists of a single trial of the entire course to travel the most tiles as possible. '''Again, the values used in the sections of code below are only examples - the actual values must be determined through experimentation.''' The averages from the best type of coding (Part 1 vs Part 2) are also used to calculate the competition ratio.
=== Procedure ===
The robot competition consists of attempting to travel the most amount of tiles possible in a single run. '''Only one trial counts as valid and must verbally be declared before the run starts.'''
# Recreate the line following code to allow the robot to travel up the ramp. Test different values of power to overcome the ramp. The end statement should cause the robot to stop '''10 cm''' away from the wall.
# Notice that everything until the second loop is a replica of the Software Trial code. You only have to add the last loop. '''Ask a TA for assistance if you don't know how to do it.'''
#: [[Image:Section_2_complete_revised.PNG|thumb|640px|center|Figure 20: Full Competition Code]]
===Competition Rules===
# Only parts provided in the EV3 kits are allowed (no regular Lego's)
# No changes in physical robot design
# No "re-tests"
# The trial used for the competition must be declared before the robot runs and cannot be disputed
# A robot counts as clearing a tile if it '''stops''' on the tile with '''at least 50%''' of the robot on the tile
===Competition Ratio===
The team with the highest competition ratio wins the competition. A<sub>distance</sub> is the robot's average distance from the wall for four trials, A<sub>angle</sub> is the robot's average angle of deviation for four trials, and N<sub>tiles</sub> is the number of tiles cleared during the single competition run:
<math> \frac{{1}}{{1 + |A_{distance} - 5|}}\ + \frac{{1}}{{1 + |A_{angle} - 360|}}\ + N_{tiles} </math>


The lab work is now complete. Please clean up the workstation. Return all unused materials to the TA. Refer to the [[#Assignment|Assignment]] section for the instructions needed to prepare the lab report.
The lab work is now complete. Please clean up the workstation. Return all unused materials to the TA. Refer to the [[#Assignment|Assignment]] section for the instructions needed to prepare the lab report.
Line 193: Line 196:
= Assignment =
= Assignment =
== Individual Lab Report ==
== Individual Lab Report ==
Follow the lab report guidelines laid out in the page called [[Specifications for Writing Your Lab Reports]] in the ''Technical Communication'' section of this manual. The following discussion points should be addressed in the appropriate section of the lab report:
Follow the lab report guidelines laid out in the page called [[Specifications for Writing Your Lab Reports]] in the ''Technical Communication'' section of this manual. The following points should be addressed in the appropriate section of the lab report.


* Discuss product evaluation and the need for it.
* Discuss product evaluation and the need for it
* Discuss quality improvement and its importance.
* Discuss quality improvement and its importance
* Explain why reverse engineering is important.
* Include examples of product evaluation and quality improvement
* Discuss a standard, an average, accuracy, and precision.
* Discuss a standard, an average, accuracy, and precision
* Discuss the EV3 kit and software used.
* Describe loops and switches and how they are useful in coding
* Assess the techniques used in the testing process.
* Describe the light sensor, gyro sensor, and ultrasonic sensor used in the lab and their optimal position
* Discuss better testing methods for the robot design.
* Discuss the importance of tolerance and percent accuracy in determining product quality
* Describe loops and switches and how they are useful in coding.
* Discuss the EV3 kit and software used
* Describe the light sensor, gyro sensor, and ultrasonic sensor used in the lab
* Describe Mindstorms programming for '''only the sensors coding portion'''
* Compare the code with sensors and code without sensors and determine which is better. Substantiate the decision with data, observations, and other facts collected.
* Compare the final code with sensors and final code without sensors; determine which is better. * * Substantiate the decision with data, observations, and other facts collected
* Make a final evaluation of the product: should it go into production? Describe its accuracy and precision.
* Make a final evaluation of the product: should it go into production? Describe its accuracy and precision
* Discuss the importance of ''%Prec'' and ''%Acc'' in determining product quality.
* Assess the techniques used in the testing process
* Recommend if any design improvements should be made.
* Discuss better testing methods for the robot design
* Recommend if any design improvements should be made


{{Lab notes}}
{{Lab notes}}
Line 213: Line 217:
== Team PowerPoint Presentation ==
== Team PowerPoint Presentation ==


Follow the presentation guidelines laid out in the page called [[EG1003 Lab Presentation Format]] in the ''Introduction to Technical Presentations'' section of this manual. When preparing the presentation, consider the following points:
Follow the presentation guidelines laid out in the page called [[EG1004 Lab Presentation Format]] in the ''Introduction to Technical Presentations'' section of this manual. When preparing the presentation, consider the following points:


* Based on the test results, what is the product's marketability?
* Based on the test results, what is the product's marketability?
* If determinations were negative, what improvements could be made?
* If determinations were that the robot(s) did not meet the standard, what improvements could be made?
* Why is product evaluation important?
* Why is product evaluation important?
* Discuss why quality improvement is important.
* Discuss why quality improvement is important
* Discuss the importance of utilizing sensors.
* Discuss the importance of utilizing sensors

Latest revision as of 01:59, 31 August 2022

Objectives

A pre-built robot will be evaluated for accuracy and precision by utilizing two different sets of code, the first without the use of sensors and the second with sensors. The robot will be tested on one of the three provided mini-courses. The data from the product evaluation of both sets of code will determine if the robot should or should not use sensors in navigating the mini-course.

Product Evaluation

Product evaluation is a common task for an engineer and for companies. Sometimes the product evaluated will be a company's product, and sometimes companies will be evaluating a competitor's design. In either case, this evaluation will use a standard (an accepted value), determine an average, and calculate accuracy and precision. [1]

Professional engineers must be sure of standards. For the purpose of this lab, the EG1004 standard will be 80% precision and accuracy. This means that the product has to achieve a percent accuracy of 80% and meet a tolerance of 1cm for distance and 10 degrees for angle. [1]

When taking measurements, the first step is to determine an average (Equation 1).

Equation 1: Average.

In this equation, Ap is the average, P1, P2, P3, ..., Pn are the results of the tests, and N is the number of tests performed.

The average is compared to the standard. Accuracy is a variable the must always be considered. The degree of accuracy required will allow the results to be verified. Engineers often work on projects that affect people's safety so it is critical that the solutions to the problems solved by technical professionals are accurate.

The comparison of the average to the standard is called accuracy, defined as Acc in Equation 2.

Equation 2: Accuracy.

In Equation 2, Ps is the standard value, and Ap is the value measured. Accuracy is always a positive number, so use the absolute value in case Ps is less than Ap.

Precision is the repeatability of a result, or how close the results are to each other, defined as Prec in Equation 3.

Equation 3: Precision.

In Equation 3, Prec is precision, Phigh is the highest data value and Plow is the lowest data value. Like accuracy, precision is always a positive number, so use an absolute value here as well.

A product can be accurate but imprecise, or precise but inaccurate (Figure 1). The goal is to achieve a combination of the highest accuracy and precision. In order for engineers to make a recommendation based on experimental test data, it is important to determine how a product compares to a standard. Engineers need these benchmarks to determine the quality of a product.

Figure 1: Accuracy and precision.

By determining the percent accuracy of an experiment, and comparing this percentage to a standard, it can be determined, based on experimental evidence, if the product has passed or failed. The equation for percent accuracy is:

Equation 4: Percent accuracy.

In Equation 4, %Acc is the percent accuracy, Ps is the standard value, and Ap is the value measured.

Precision shows how precise a product is compared to the standard.

Adequate precision is determined by an acceptable tolerance, set by the manufacturer. In this lab, the precision of the robot must be compared to the tolerance allowed:

Distance Test:
Angle of Deviation Test:

Calculating percent accuracy and precision and then comparing those results to a standard determines the quality of the product.

Quality Improvement

Quality improvement is a process of analyzing a design and testing it through physical modeling, computer simulation, or mathematical modeling. Quality improvement ensures that a product will perform as expected and allows improvements to be made to the design.

When deciding how to improve the design in this lab, keep these questions in mind:

  • Does the robot perform to standard?
    • If no, what can be done to improve its functionality?
    • If yes, what can be done to improve its performance beyond the standard?

Changes will be implemented after testing the initial code.

EV3 Sensors

Sensor Basics

A sensor is a device that measures a specific value and is used as an input. Different properties can be measured by the EV3 sensors, such as light intensity, angle, or distance. The three sensors used in this lab are the light sensor, gyro sensor, and an ultrasonic sensor.

Light Sensor

The light sensor or color sensor (Figure 6) can measure reflected light intensity, color, and ambient light, and is useful when the robot has the option to follow a line or use differences in color to perform different actions, such as stopping when detecting a different color. If measuring color, the sensor outputs a value (0-7) that corresponds to a specific color. Measuring reflected light outputs a value (0-100) depending on the intensity of the light that is reflected. The ambient light works the same way as the reflected light, except it measures only the surrounding light. The light sensor, as well as all other sensors, can be found in the yellow category the Mindstorms software. The light sensor should be placed so that it is 2 cm, at max, away from whatever it trying to record.

Figure 6: Light sensor.

The type of value the sensor can measure can be changed by clicking the small box on the bottom left of the block (Figure 7).

Figure 7: Light sensor, value input.

Gyro Sensor

The gyro sensor can measure the degrees it turns or the rate at which it is turning. For example, if the robot is set to turn 90º using the gyro, the robot will keep trying to turn until 90º is reached, regardless of any bumps in its way (Figure 8). The gyro sensor should be placed parallel to the plane of the EV3 Brick.

Figure 8: Gyro sensor.


Ultrasonic Sensor

The ultrasonic sensor measures the distance between the robot and the closest object to it. It can measure distance in both centimeters and inches; the recommended range is 1 cm to 250 cm (Figure 9).

Figure 9: Ultrasonic sensor.

Switches and Loops

Switches and loops are used to tell the robot what actions to perform depending on the inputs from its surroundings, measured by the sensors.

Switch

A switch is an “if-else” statement: if the initial condition is true, the top action will occur, if false, the bottom action will occur. This block is used when the robot has to perform different actions depending on the input from the sensors (Figure 10).

Figure 10: Switch.

Loop

A switch will only run the code once, so if repeated action is desired, a loop must be used to repeat the switch’s statement. A loop continuously runs the code until the end statement occurs. The end statement can be determined with a sensor or hard-coded to a specific duration or rotations. Figure 11 shows an infinite loop.

Figure 11: Loop.

Materials and Equipment

  • A Mindstorms kit with pre-made robot
  • A computer with Mindstorms software
  • A mini-course with accuracy markings

Procedure

Problem Statement

Evaluate two types of code for a robot by creating a line of code without the use of sensors, and then a separate line of code using sensors. For each trial, record the distance from the wall and angle of deviation from the center line. Test the first code by running four trials of Section 1 of the mini-course as marked on the physical course. Create a second line of code using switches and loops that incorporate the types of sensors found on the robot and features on the mini-course. Test the robot again with four trials of Section 1. This part of the procedure will have a total of 8 trials. Download the pre-made Datasheet and fill in the data for each trial.

Course Breakdown

The mini-course (Figure 12) has a designated Section 1 to complete the eight trials of the lab: one flat straight section of four tiles and a right turn with a wall at the end. The competitive trial will utilize the entire course. In Figure 8, Section 1 is marked in red.

Figure 12: Mini-course.

Part 1: Hardcoding

Based off of your observations, make design changes to the robot in order to run trials on the first section (Section 1) of the mini-course.

Every run, unless the robot goes off the course, counts as a trial. There is only one test run allowed after which all runs will be recorded as trials. When troubleshooting the code, please cross check the port number on the code and the port the sensor/motor is connected to before anything else.

Product Evaluation

Using the modified design, conduct four trials of the robot using a code without sensors to travel in a straight line across the first four tiles, turn 90 degrees to the right, and stop 5 cm from the wall. The values used in the sections of code below are only examples - the actual values must be determined through experimentation.

  1. Start the Mindstorms EV3 software.
  2. Download the zipped folder containing the Mindstorms program.
  3. The hardcoding program should look like the one pictured below:
    Figure 17: Part 1 Code
  4. Please read the comments before running the program.
  5. Change the ports to the configuration on your robot.
  6. Run a sample trial to understand the code.
  7. Evaluate the number of rotations by measuring the total distance that the robot has to move for a certain block of code and divide that by the perimeter of the wheel.
  8. Change the values to the evaluated numbers.
  9. Try to stop 5cm before the wall.
  10. Run the program for four trials. For each trial, record the distance and angle using pre-made markings on the sides of the tile.
  11. Distance is measured as the distance from the wall for which the ruler on the side of the tile can be used. (Denote it as y)
  12. Angle of deviation is calculated in the following way:
    1. Measure deviation from the center of the wall. (Denote it as x)
    2. tan^-1 (x/y)
    3. If you are confused as to how to calculate the angle of deviation, ask your TA for help.

Part 2: Sensors

Every run, unless the robot goes off the course, counts as a trial. There is only one test run allowed after which all runs will be recorded as trials. When troubleshooting the code, please cross check the port number on the code and the port the sensor/motor is connected to before anything else.

Quality Improvement

Using the results of the original robot, incorporate the use of sensors in the robot's code. The following steps outline how to build a line-following code for the robot to stay on track. The values used in the sections of code below are only examples; the actual values must be determined through experimentation.

  1. Move to the Software Trials tab
  2. The program should look like this:
    Figure 18: Software Trials Code
  3. Please read the comments before running the program
  4. Change the ports to the configuration on your robot
  5. Run a sample trial to understand the code
  6. Using the port view shown below, set the readings on the Mindstorms code for the light sensors and the gyro sensors blocks to the appropriate values. Adjust Rotations as done in the previous part. Ask a TA for assistance if you don't know what values to use.
    Figure 19: Port view
  7. Take the same measurements as for Part 1: Harcoding
  8. Tabulate results neatly in the Datasheet

Tabulation of Results

All data must be inputted into the Datasheet. From the datasheet, determine if the EG standards were met for accuracy and precision; recorded precision must be divided by 2 to get the corresponding tolerance:

Hardcode Accuracy %Acc Pass/Fail Precision Tolerance Pass/Fail
Distance Test
Angle of Deviation Test


Sensors Code Accuracy %Acc Pass/Fail Precision Tolerance Pass/Fail
Distance Test
Angle of Deviation Test

Part 3: Robot Competition

The robot competition consists of a single trial of the entire course to travel the most tiles as possible. Again, the values used in the sections of code below are only examples - the actual values must be determined through experimentation. The averages from the best type of coding (Part 1 vs Part 2) are also used to calculate the competition ratio.

Procedure

The robot competition consists of attempting to travel the most amount of tiles possible in a single run. Only one trial counts as valid and must verbally be declared before the run starts.

  1. Recreate the line following code to allow the robot to travel up the ramp. Test different values of power to overcome the ramp. The end statement should cause the robot to stop 10 cm away from the wall.
  2. Notice that everything until the second loop is a replica of the Software Trial code. You only have to add the last loop. Ask a TA for assistance if you don't know how to do it.
    Figure 20: Full Competition Code

Competition Rules

  1. Only parts provided in the EV3 kits are allowed (no regular Lego's)
  2. No changes in physical robot design
  3. No "re-tests"
  4. The trial used for the competition must be declared before the robot runs and cannot be disputed
  5. A robot counts as clearing a tile if it stops on the tile with at least 50% of the robot on the tile

Competition Ratio

The team with the highest competition ratio wins the competition. Adistance is the robot's average distance from the wall for four trials, Aangle is the robot's average angle of deviation for four trials, and Ntiles is the number of tiles cleared during the single competition run:

The lab work is now complete. Please clean up the workstation. Return all unused materials to the TA. Refer to the Assignment section for the instructions needed to prepare the lab report.

Assignment

Individual Lab Report

Follow the lab report guidelines laid out in the page called Specifications for Writing Your Lab Reports in the Technical Communication section of this manual. The following points should be addressed in the appropriate section of the lab report.

  • Discuss product evaluation and the need for it
  • Discuss quality improvement and its importance
  • Include examples of product evaluation and quality improvement
  • Discuss a standard, an average, accuracy, and precision
  • Describe loops and switches and how they are useful in coding
  • Describe the light sensor, gyro sensor, and ultrasonic sensor used in the lab and their optimal position
  • Discuss the importance of tolerance and percent accuracy in determining product quality
  • Discuss the EV3 kit and software used
  • Describe Mindstorms programming for only the sensors coding portion
  • Compare the final code with sensors and final code without sensors; determine which is better. * * Substantiate the decision with data, observations, and other facts collected
  • Make a final evaluation of the product: should it go into production? Describe its accuracy and precision
  • Assess the techniques used in the testing process
  • Discuss better testing methods for the robot design
  • Recommend if any design improvements should be made

Remember: Lab notes must be taken. Experimental details are easily forgotten unless written down. EG1004 Lab Notes Paper can be downloaded and printed from the EG1004 Website. Use the lab notes to write the Procedure section of the lab report. At the end of each lab, a TA will scan the lab notes and upload them to the Lab Documents section of the EG1004 Website. One point of extra credit is awarded if the lab notes are attached at the end of the lab report. Keeping careful notes is an essential component of all scientific practice.

Team PowerPoint Presentation

Follow the presentation guidelines laid out in the page called EG1004 Lab Presentation Format in the Introduction to Technical Presentations section of this manual. When preparing the presentation, consider the following points:

  • Based on the test results, what is the product's marketability?
  • If determinations were that the robot(s) did not meet the standard, what improvements could be made?
  • Why is product evaluation important?
  • Discuss why quality improvement is important
  • Discuss the importance of utilizing sensors
  1. ^ a b Oakes, W.C., L.L. Leone, and C.G. Gunn, Engineering Your Future, MI: Great Lakes Press, 2002.