Difference between revisions of "Product Evaluation and Quality Improvement (EV3)"

From EG1004 Lab Manual
Jump to: navigation, search
(1003 --> 1004)
 
(11 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= Objectives =
= Objectives =
The objectives of this lab are to use product evaluation to determine if a robot design meets minimum standards for accuracy and precision, to use a quality improvement process to modify the robot design, to evaluate the modified robot design, and to use the data from the product evaluation of both robots to determine if the original robot or the modified robot should go into production.
The objectives of this lab are to use product evaluation to determine if a robot design meets minimum standards for accuracy and precision, to use a quality improvement process to modify the robot design, to evaluate the modified robot design, and to use the data from the product evaluation of both robots will determine if the original robot or the modified robot should go into production.


= Overview =
= Overview =
Line 8: Line 8:
Product evaluation is a common task for an engineer and for companies. Sometimes the product evaluated will be a company's product, and sometimes companies will be evaluating a competitor's design. In either case, this evaluation will use a '''''standard''''' (an accepted value), determine an '''''average''''', and calculate '''''accuracy''''' and '''''precision'''''.<ref name="one">Oakes, W.C., L.L. Leone, and C.G. Gunn, ''Engineering Your Future'', MI: Great Lakes Press, 2002.</ref>
Product evaluation is a common task for an engineer and for companies. Sometimes the product evaluated will be a company's product, and sometimes companies will be evaluating a competitor's design. In either case, this evaluation will use a '''''standard''''' (an accepted value), determine an '''''average''''', and calculate '''''accuracy''''' and '''''precision'''''.<ref name="one">Oakes, W.C., L.L. Leone, and C.G. Gunn, ''Engineering Your Future'', MI: Great Lakes Press, 2002.</ref>


Professional engineers must be sure of standards. For the purpose of this lab, the EG1003 Standard will be 80% precision and accuracy. This means that the product has to achieve a percent accuracy and percent precision of 80 percent and it has to pass 80 percent of the tests that it undergoes.<ref name="one"></ref>
Professional engineers must be sure of standards. For the purpose of this lab, the EG1004 standard will be 80% precision and accuracy. This means that the product has to achieve a percent accuracy and percent precision of 80and it has to pass 80of the tests that it undergoes.<ref name="one"></ref>


When taking measurements, the first step is to determine an average.
When taking measurements, the first step is to determine an average (Equation 1).


<math>Average\left(A_p\right) = \frac{P_1+P_2+P_3+...+P_n}{N}\,</math>
<math>Average\left(A_p\right) = \frac{P_1+P_2+P_3+...+P_n}{N}\,</math>


In this equation, ''A<sub>p</sub>'' is the average, ''P<sub>1</sub>'', ''P<sub>2</sub>'', ''P<sub>3</sub>'', ''...'', ''P<sub>n</sub>'' are the results of the tests and ''N'' is the total number of tests performed.
Equation 1: Average.


The '''''average''''' is compared to the '''''standard'''''. Accuracy is a variable the must always be considered. The degree of accuracy required will allow the results to be verified. Engineers often work on projects that affect people's safety; it is critical that the solutions to the problems solved by technical professionals are '''''accurate'''''.
In Equation 1, ''A<sub>p</sub>'' is the average, ''P<sub>1</sub>'', ''P<sub>2</sub>'', ''P<sub>3</sub>'', ''...'', ''P<sub>n</sub>'' are the results of the tests, and ''N'' is the number of tests performed.
 
The '''''average''''' is compared to the '''''standard'''''. Accuracy is a variable that must always be considered. The degree of accuracy required will allow the results to be verified. Engineers often work on projects that affect people's safety so it is critical that the solutions to the problems solved by technical professionals are accurate.


The comparison of the average to the standard is called accuracy, defined as ''Acc'' in the following equation:
The comparison of the average to the standard is called accuracy, defined as ''Acc'' in the following equation:
Line 22: Line 24:
<math>Acc = \left|P_s - A_p\right|\,</math>
<math>Acc = \left|P_s - A_p\right|\,</math>


In this equation, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured. Accuracy is always a positive number, so use the absolute value in case ''P<sub>s</sub>'' is less than ''A<sub>p</sub>''.
Equation 2: Accuracy.


Precision is the repeatability of a result, or how close the results are to each other, defined as ''Prec'' in the following equation:
In Equation 2, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured. Accuracy is always a positive number so use the absolute value in case ''P<sub>s</sub>'' is less than ''A<sub>p</sub>''.
 
Precision is the repeatability of a result, or how close the results are to each other, defined as ''Prec'' in Equation 3:


<math>Prec = \left|P_{high} - P_{low}\right|\,</math>
<math>Prec = \left|P_{high} - P_{low}\right|\,</math>


In this equation, ''Prec'' is precision, ''P<sub>high</sub>'' is the highest data value and ''P<sub>low</sub>'' is the lowest data value. Like accuracy, precision is always a positive number, so use an absolute value here as well.
Equation 3: Precision.


A product test can be accurate but imprecise, or precise but inaccurate (Figure 1). The goal is always the combination of accuracy and precision. In order for engineers to make a recommendation based on experimental test data, it is important to determine how a product compares to a standard. Engineers need these benchmarks to determine the quality of a product.
In Equation 3, ''Prec'' is precision, ''P<sub>high</sub>'' is the highest data value and ''P<sub>low</sub>'' is the lowest data value. Like accuracy, precision is always a positive number so use an absolute value.
 
A product can be accurate but imprecise, or precise but inaccurate (Figure 1). The goal is to achieve a combination of the highest accuracy and precision. For engineers to make a recommendation based on experimental test data, it is important to determine how a product compares to a standard. Engineers need these benchmarks to determine the quality of a product.


[[Image:Lab hardsyn 12.gif|frame|center|Figure 1: Accuracy and precision.]]
[[Image:Lab hardsyn 12.gif|frame|center|Figure 1: Accuracy and precision.]]


By determining the percent accuracy of an experiment, and comparing this percentage to a standard, it can be determined, based on experimental evidence, if the product has passed or failed. The equation for percent accuracy is:
By determining the percent accuracy of a product and comparing this percentage to a standard, the experimental evidence determines if the product has passed or failed (Equation 4).  


<math>%Acc = \left ( 1 - \frac{ \left | P_s - A_p \right | }{P_s} \right ) \times 100%\,</math>
<math>%Acc = \left ( 1 - \frac{ \left | P_s - A_p \right | }{P_s} \right ) \times 100%\,</math>


In this equation, ''%Acc'' is the percent accuracy, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured.
Equation 4: Percent accuracy.


Percent accuracy shows how precise a product is compared to the standard. The equation for percent precision is:
In Equation 4, ''%Acc'' is the percent accuracy, ''P<sub>s</sub>'' is the standard value, and ''A<sub>p</sub>'' is the value measured.
 
Percent accuracy shows how precise a product is compared to the standard (Equation 5).  


<math>%Prec = 100e^{-BP}\,</math>
<math>%Prec = 100e^{-BP}\,</math>


In this equation, ''%Prec'' is the percent precision, ''B'' is the exponential decay factor, and ''P'' is the precision measured. Each individual test requires the use of a different exponential decay factor (''B''):
Equation 5: Percent precision.
 
In Equation 5, ''%Prec'' is the percent precision, ''B'' is the exponential decay factor, and ''P'' is the precision measured. Each test requires the use of a different exponential decay factor (''B''):
: Distance Test: ''B'' = 0.0972
: Distance Test: ''B'' = 0.0972
: Angle of Deviation Test: ''B'' = 0.0323
: Angle of Deviation Test: ''B'' = 0.0323
Line 51: Line 61:


== Quality Improvement ==
== Quality Improvement ==
Quality improvement is a process of analyzing a design and testing it either through physical modeling, computer simulation, or mathematical modeling. Quality improvement ensures that a product will perform as expected and allows improvements to be made to the design.
Quality improvement is a process of analyzing a design and testing it through physical modeling, computer simulation, or mathematical modeling. Quality improvement ensures that a product will perform as expected and allows improvements to be made to the design.


When deciding how to improve the design in this lab, keep these questions in mind:
When deciding how to improve the design in this lab, keep these questions in mind:
Line 61: Line 71:


== Reverse Engineering ==
== Reverse Engineering ==
Reverse engineering software involves reversing a program's machine code to obtain the original source code or determine the file structures the program uses. This needs to be done when source code is lost or is not available. For example, when a project is reactivated after being dormant for years to add new features to a product, the original design documentation may have disappeared. Also, when companies are writing a program that must interface to a second company's software, the second company may not have any incentive to cooperate by describing how the data in the files they use are organized, forcing the first company to deduce it on their own. Sometimes, this process is also under­taken as a way to improve the performance of a program, to fix a bug, or to find a virus. When the source code is obtained in this way for any of these reasons it is legal and necessary. Reverse engineering software in order to copy it constitutes a copy­right violation and is illegal.<ref name="two">''What Is ''website. TechTarget Network. Retrieved July 29th, 2003.</ref>
Reverse engineering software involves reversing a program's machine code to obtain the original source code or determine the file structures the program uses. This is done when source code is lost or is not available. For example, when a project is reactivated after being dormant for years to add new features to a product, the original design documentation may have been lost. Also, when companies are writing a program that must interface with a second company's software, the second company may not have any incentive to cooperate by describing how the data in the files they use are organized, forcing the first company to deduce it on their own. Sometimes, this process is also under¬taken as a way to improve the performance of a program, to fix a bug, or to find a virus. When the source code is obtained in this way for any of these reasons, it is legal and necessary. Reverse engineering software to copy it constitutes a copy¬right violation and is illegal.<ref name="two">''What Is ''website. TechTarget Network. Retrieved July 29th, 2003.</ref>


Hardware reverse engineering involves taking apart a device to see how it works. If a processor manufacturer wants to see how a competitor's processor works, the company can purchase the processor, disassemble it, and then make a new processor similar to it. In some countries, this process is illegal. Hardware reverse engineering is quite expensive and requires an expert in the field.<ref name="two"></ref>
Hardware reverse engineering involves taking apart a device to see how it works. If a processor manufacturer wants to see how a competitor's processor works, the company can purchase the processor, disassemble it, and make a new processor similar to it. In some countries, this process is illegal. Hardware reverse engineering is quite expensive and requires an expert in the field.<ref name="two"></ref>


When beginning the process of reverse engineering the robot, identifying the gear trains and the individual gears used in the original design and in the final design is critical to understanding the robot's operation.
When beginning the process of reverse engineering the robot, identifying the gear trains and the individual gears used in the original design and in the final design is critical to understanding the robot's operation.


There are two types of gear trains: simple (Figure 2) and compound (Figure 3). A simple gear train has its gears arranged in a line (see Figure 1).
There are two types of gear trains: simple (Figure 2) and compound (Figure 3). A simple gear train has its gears arranged in a line (Figure 3).


[[Image:Lab reveng 1.jpg|frame|center|Figure 2: Simple gear train.]]
[[Image:Lab reveng 1.jpg|frame|center|Figure 2: Simple gear train.]]
Line 75: Line 85:
[[Image:Lab reveng 2.jpg|frame|center|Figure 3: Compound gear train.]]
[[Image:Lab reveng 2.jpg|frame|center|Figure 3: Compound gear train.]]


Gear trains have measurable characteristics known as gear ratio and velocity ratio. These characteristics are inversely proportional. Gear ratio equals output over input, while velocity ratio equals input over output. If multiple gears are being used, the gear ratio and velocity ratio are the product of these ratios for each individual gear. To determine input and output values, engineers count the number of teeth on each gear or by measuring each gear's radius.
Gear trains have measurable characteristics known as a gear ratio and a velocity ratio. These ratios are inversely proportional. Gear ratio equals output over input, and velocity ratio equals input over output. If multiple gears are being used, the gear ratio and velocity ratio are the product of these ratios for each individual gear. To determine input and output values, engineers count the number of teeth on each gear or by measure each gear's radius.


For example, to compute the gear ratio of the gear trains in Figures 2 and 3, these formulas (Figure 4) are used, where the values are the radii of the gears in the gear train:
For example, to compute the gear ratio of the gear trains in Figures 2 and 3, these formulas (Figure 4) are used, where the values are the radii of the gears in the gear train.


{| class="wikitable" style="text-align: center; margin: 0px auto;"
{| class="wikitable" style="text-align: center; margin: 0px auto;"
Line 85: Line 95:
|}
|}


For example, to compute the velocity ratio of the gear trains above, these formulas (Figure 5) are used:
For example, to compute the velocity ratio of the gear trains in Figures 2 and 3, these formulas (Figure 5) are used.
 
{| class="wikitable" style="text-align: center; margin: 0px auto;"
{| class="wikitable" style="text-align: center; margin: 0px auto;"
|+ style="caption-side: bottom;" | Figure 5: Velocity ratio.
|+ style="caption-side: bottom;" | Figure 5: Velocity ratio.
Line 109: Line 118:
The objective of reverse engineering the robot is to understand how it works and to calculate the gear and velocity ratios for its drive train (remember the wheels are part of the drive train). Remember that engineers reverse engineer products to reproduce or improve them. As the robot is disassembled, improvements to the robot's design should be considered.
The objective of reverse engineering the robot is to understand how it works and to calculate the gear and velocity ratios for its drive train (remember the wheels are part of the drive train). Remember that engineers reverse engineer products to reproduce or improve them. As the robot is disassembled, improvements to the robot's design should be considered.


'''Note''': Using gear and velocity ratios have proven to be extremely helpful when building robots for your EG1003 Semester-Long Design Project.
'''Note''': Using gear and velocity ratios have proven to be extremely helpful when building robots for the EG1004 Semester-Long Design Project.
 
= Assignment =
 
== Individual Lab Report ==
 
Follow the lab report guidelines laid out in the page called [[Specifications for Writing Your Lab Reports]] in the ''Technical Communication'' section of this manual. The following discussion points should be addressed in the appropriate section of the lab report:


* Discuss product evaluation and the need for it.
= Design Considerations =
* Discuss quality improvement and its importance.
Carefully consider the lab objectives and what needs to be done to achieve them:
* Explain why reverse engineering is important.
* What made or will make the results inaccurate and imprecise?
* Discuss a standard, an average, accuracy, and precision.
* If the robot failed a test, why did it fail?
* Discuss the EV3 kit and software used.
* Assess the techniques used in the testing process.
* Discuss better testing methods for the robot design.
* Describe how the components fit together and how the device was constructed.
* Describe and show how the gear and velocity ratios were calculated, and discuss their importance in the overall robot design as well.
* Analyze the original robot design and the final design then make a decision which design is better. Remember to substantiate the decision with the data, observations, and facts collected.
* Make a final evaluation of the product: should it go into production? Describe its accuracy and precision.
* Discuss the importance of ''%Prec'' and ''%Acc'' in determining product quality.
* Recommend if any design improvements should be made.
 
== Team PowerPoint Presentation ==
 
Follow the presentation guidelines laid out in the page called [[EG1003 Lab Presentation Format]] in the ''Introduction to Technical Presentations'' section of this manual. When preparing the presentation, consider the following points:
 
* Based on the test results, what is the product's marketability?
* If determinations were negative, what improvements could be made?
* Why is product evaluation important?
* Discuss why quality improvement is important.
* Discuss the importance of gear and velocity ratios.


= Materials and Equipment =
= Materials and Equipment =


* Mindstorms kit
* A Mindstorms kit
* Computer with Mindstorms software
* A computer with Mindstorms software
* Protractor and ruler
* A crotractor and a ruler
* Graph paper
* Graph paper
'''Remember:''' Lab notes are required. Experimental details are easily forgotten unless written down. EG Standard Note Paper can be downloaded and printed from here: [https://eg.poly.edu/downloads/Note_paper.zip https://eg.poly.edu/downloads/Note_paper.zip]. Use the lab notes to write the Procedure section of the lab report. At the end of each lab, the TA will scan the lab notes and upload them to [https://eg.poly.edu https://eg.poly.edu]. The lab notes must be attached at the end of the lab report (use the "Insert Object" command in MS Word after the Conclusion). Keeping careful notes is an essential component of all scientific practice.


= Procedure =
= Procedure =
Line 155: Line 137:
Evaluate a robot design by building a specified robot design. Test the robot design, make improvements on the design, and retest it. While taking it apart, analyze the drive train.
Evaluate a robot design by building a specified robot design. Test the robot design, make improvements on the design, and retest it. While taking it apart, analyze the drive train.


# Build a robot according to the design assigned by the TA.
# Build a robot according to the design assigned by the TA
# Start the Mindstorms EV3 software.
# Start the Mindstorms EV3 software
# Click on the '''+''' in the top left corner to open up a blank program.
# Click on the '''+''' in the top left corner to open up a blank program
#: [[Image:Lab evalimprov ev3 2.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 2.png|thumb|center|640px|Figure 6: Mindstorms start page.]]
# The screen should look like this. Your program begins with the green play button.
# The screen should look like the screen shot in Figure 6. The program begins with the Start block (Figure 7)
#: [[Image:Lab evalimprov ev3 3.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 3.png|thumb|center|640px|Figure 7: Start block.]]
# Place a Large Motor block from the Action category next to the Start block. Along the bottom of the Large Motor block, there are four options. From left to right, they are Control Mode, Power, Rotations, and Brake at End.
# 5. Place a Large Motor block from the Action palette next to the Start block. Along the bottom of the Large Motor block, there are four options (Figure 8). On the left, there is a Mode Selector button followed by three values for the input parameters picture above the value: from left to right, Power, Rotations, and Brake at End
#: [[Image:Lab evalimprov ev3 4.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 4.png|thumb|center|640px|Figure 8: Motor block.]]
# Change the Control Mode of the Large Motor block to ''On for Seconds''. This will change the Rotations input to Seconds.
# Click on the Mode Selector (Figure 9) and change the mode of the Large Motor block to ''On for Seconds''. This will change the Rotations input parameter to Seconds
#: [[Image:Lab evalimprov ev3 5.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 5.png|thumb|center|640px|Figure 9: Mode selector.]]
# Set Power to 100 and Seconds to 5. Using these settings, the motor will move at 100% power for five seconds and then brake to a stop.
# Set Power to 100 and Seconds to five (Figure 10) . Using these values, the motor will move at 100% power for five seconds and then brake to a stop
#: [[Image:Lab evalimprov ev3 6.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 6.png|thumb|center|640px|Figure 10]]
# Assign the port that the motor is connected to at the top right corner of the block. Assume our motor is connected to Port A.
# 8. Assign the port that the motor is connected to at the top right corner of the block. Assume the motor is connected to Port A (Figure 11)
#: [[Image:Lab evalimprov ev3 7.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 7.png|thumb|center|640px|Figure 11: Port assignment.]]
# Place a Wait block from the Flow Control category after the Large Motor block. By default, the Wait block will wait for one second. Change the Seconds input to 3 so that there is a three second pause in the execution of the program.
# 9. Place a Wait block from the Flow Control palette after the Large Motor block. By default, the Wait block will wait for one second. Change the Seconds input parameter value to three so that there is a three second pause in the execution of the program (Figure 12)
#: [[Image:Lab evalimprov ev3 8.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 8.png|thumb|center|640px|Figure 12: Wait block.]]
# Select an Invert Motor block from the Advanced category and position it after the Wait block. By default, the Invert input is set to True. Following the execution of this block, motors will move in reverse until another Invert Motor block is executed with the Invert input set to False.
# Select an Invert Motor block from the Advanced palette and position it after the Wait block. By default, the Invert parameter is set to True (Figure 13). Following the execution of this block, motors will move in reverse until another Invert Motor block is executed with the Invert parameter set to False
#: [[Image:Lab evalimprov ev3 9.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 9.png|thumb|center|640px|Figure 13: Invert parameter.]]
# Take another Large Motor block from the Action category and place it after the Invert Motor block. Set Control Mode to ''On for Seconds'', Power to 100, Seconds to 5, and Port to A.
# Take another Large Motor block from the Action palette and place it after the Invert Motor block. Set Mode to On for Seconds, Power to 100, Seconds to five, and Port to A (Figure 14)
#: [[Image:Lab evalimprov ev3 10.png|thumb|center|640px]]
#: [[Image:Lab evalimprov ev3 10.png|thumb|center|640px|Figure 14: Completed program.]]
# Run the program. The program should run the motor connected to Port A for five seconds at 100% power, wait for three seconds, then run the motor connected to Port A in reverse for fives seconds at 100% power.
# Run the program. The program should run the motor connected to Port A for five seconds at 100% power, wait for three seconds, then run the motor connected to Port A in reverse for five seconds at 100% power.


'''Note:''' Program and test the robot for each test specified. To do this, make a list of the tests when taking notes. Brainstorm what the robot should do in each test. Then, use the notes to write the program. Record the standard, average, accuracy, and precision. If the robot does not operate correctly, troubleshoot the problem. The TA must initial the original data.
Program and test the robot for each test specified. To do this, make a list of the tests. Brainstorm what the robot should do in each test. Then, write the program. Record the standard, average, accuracy, and precision. If the robot does not operate correctly, troubleshoot the problem. The TA must initial the original data.
 
'''Note:''' The initial program cannot be used for the following tests.


== Distance Test ==
== Distance Test ==
'''Note:''' Use centimeters for this test.''
'''Note:''' Use centimeters for this test.''


In this test, the robot will go forward a set distance in a specified time. Test the robot's accuracy and precision for this distance. Before testing begins, the standard must be found. To find the standard, program the robot to go forward for five seconds and measure the distance it travels. Divide the result by five, and then multiply by four. This number is the standard.
In this test, the robot will go forward a set distance in a specified time. Test the robot's accuracy and precision for distance. Before testing begins, the standard must be found. To find the standard, program the robot to go forward for five seconds and measure the distance it travels. Divide the result by five, and then multiply by four. This result is the standard for distance.


<math>P_s = \frac{distance\ traveled\ in\ 5\ seconds}{5} \times 4\,</math>
<math>P_s = \frac{distance\ traveled\ in\ 5\ seconds}{5} \times 4\,</math>
Equation 6: Distance test, standard formula.


Once the standard has been obtained, the testing can begin.
Once the standard has been obtained, the testing can begin.


[[Image:ProdEval1.png|thumb|480px|right|Figure 4: Sample drawing paper with data. The green line represents the five-second standard measurement. The blue lines represent the five four-second trials.]]
[[Image:ProdEval1.png|thumb|480px|right|Figure 15: Sample drawing paper with data. The green line represents the five-second standard measurement. The blue lines represent the five four-second trials.]]
'''Warning:''' Read the section on the [[#Angle of Deviation Test|Angle of Deviation Test]] before performing this test.
'''Warning:''' Read the section on the [[#Angle of Deviation Test|Angle of Deviation Test]] before performing this test.


# Upload a program that will prompt the robot to travel for 4 seconds.
# Upload a program that will prompt the robot to travel for four seconds
# Measure the distance traveled.
# Measure the distance traveled
# Repeat this test five times.
# Repeat this test five times


== Angle of Deviation Test ==
== Angle of Deviation Test ==
'''Note:''' This test is performed at the same time as the Distance Test. Since the robot is expected to travel in a straight line, the standard is 360&deg;.
'''Note:''' This test is performed at the same time as the distance test. Since the robot is expected to travel in a straight line, the standard is 360&deg;.


<math>P_s = 360^\circ\,</math>
<math>P_s = 360^\circ\,</math>


In this test, the robot will move forward for four seconds and the angle of deviation will be recorded. The angle of deviation is the angle between a straight line and a line drawn from the starting point to where the robot finishes.
Figure 16: Standard for the angle of deviation test.


# Use a protractor to draw a 0&deg; line from the start point.
In this test, the robot will move forward for four seconds and the angle of deviation will be recorded. The angle of deviation is the angle between a center line or a 0º line and a line drawn from the starting point to where the robot finishes.
# Mark the start point and place the robot there. Identify a reference point on the robot.
 
# Run the Distance Test.
# Use a protractor to draw a 0&deg; line from the start point
# Make a mark at the point where the robot finished, and draw a line from the start point to this mark.
# Mark the start point and place the robot there. Identify a reference point on the robot
# Measure the positive angle of deviation.
# Run the distance test
# Repeat this procedure five times.
# Make a mark at the point where the robot finished, and draw a line from the start point to this mark
# Measure the positive angle of deviation
# Repeat this procedure five times
<br style="clear: both;" />
<br style="clear: both;" />


== Quality Improvement ==
== Quality Improvement ==
# Assess the robot design from data collected and from observations.
# Assess the robot design using the data collected and from observations
# Make necessary adjustments.
# Make necessary adjustments
# Retest the robot and collect data.
# Retest the robot and collect data
# Take pictures of both the original design and the modified design.
# Take pictures of both the original design and the modified design


== Reverse Engineering ==
== Reverse Engineering ==
# Visually assess the robot's design. Consider how it works.
# Visually assess the robot's design. Consider how it works
# Sketch the front, top, and most detailed side of the robot. Be sure to include dimensions in the sketch.
# Sketch the front, top, and most detailed side of the robot. Be sure to include dimensions in the sketch
# Also sketch the most detailed view of the gear train.
# Also sketch the most detailed view of the gear train
# Before disassembling the robot, ask the TA to take a picture of it.
# Before disassembling the robot, ask the TA to take a picture of it
# Disassemble the robot. Analyze its inner workings and sketch the gear train(s). Note: All sketches in EG1003 must be done in pencil.
# Disassemble the robot. Analyze its inner workings and sketch the gear train(s). Note: All sketches in EG1004 must be done in pencil
# Have all sketches and original data signed by the TA.
# Have all sketches and original data signed by the TA


With measurements for both tests performed and an analysis of the drive train of the robot, data analysis must begin. Calculate the average, accuracy, and precision for each test. Compare the results to the standard in order to determine if the robot is accurate and precise. Label all major components and describe their functions. Calculate percent accuracy and percent precision. Calculate the gear and velocity ratios.  
With measurements for both tests performed and an analysis of the drive train of the robot, begin data analysis. Calculate the average, accuracy, and precision for each test. Compare the robots’ percent accuracy and percent precision to the standard to determine if the robot is accurate and precise. Label all major components and describe their functions. Calculate percent accuracy and percent precision. Calculate the gear and velocity ratios.  


:'''Examples'''
:'''Examples'''
:* If ''P<sub>s</sub>'' was 5cm and ''A<sub>p</sub>'' was 7cm, then the percent accuracy (''%Acc'') is 60%. This test does not pass the 80% criterion.
:* If ''P<sub>s</sub>'' was five cm and ''A<sub>p</sub>'' was seven cm, then the percent accuracy (''%Acc'') is 60%. This test does not pass the 80% standard
:* If ''P<sub>s</sub>'' was 5cm and ''A<sub>p</sub>'' was 25cm, then the result of the percent accuracy equation is -300%. Since a negative percentage has no meaning in engineering, it can be said that the product has a percent accuracy of zero. Any percent accuracy less than zero is equal to zero.
:* If ''P<sub>s</sub>'' was five cm and ''A<sub>p</sub>'' was 25cm, then the percent accuracy is -300%. Since a negative percentage has no meaning in engineering, it can be said that the product has a percent accuracy of zero. Any percent accuracy less than zero is equal to zero
:* If ''P<sub>s</sub>'' was 10cm and ''A<sub>p</sub>'' was 8cm, then the percent accuracy is exactly 80%, which is a passing result.
:* If ''P<sub>s</sub>'' was 10cm and ''A<sub>p</sub>'' was eight cm, then the percent accuracy is 80%, which is a passing result
:* For the Distance Test, if the precision (''P'') was 0.5cm, the percent precision (''%Prec'') is 95.3%. This robot passes the 80% criterion for this test.
:* For the distance test, if the precision (''P'') was 0.5cm, the percent precision (''%Prec'') is 95.3%. This robot passes the 80% standard.
:* For the Angle of Deviation Test, if the precision (''P'') was 10.25°, the percent precision (''$Prec'') is 71.8%. This robot does not pass the 80% criterion.
:* For the angle of deviation test, if the precision (''P'') was 10.25°, the percent precision is 71.8%. This robot does not pass the 80% standard.


== Tabulation of Results ==
== Tabulation of Results ==
To decide if the product is ready to manufacture, arrange the data in a table designed to calculate its precision and accuracy. Remember it must pass all the tests at a rate of at least 80%.
To decide if the product is ready to manufacture, arrange the data in a table designed to calculate its precision and accuracy. Remember it must pass all the tests by at least 80%.


{| class="wikitable" style="text-align: center; margin: 0px auto;"
{| class="wikitable" style="text-align: center; margin: 0px auto;"
Line 244: Line 232:
|}
|}


Retest with new design using the same table, record the data.
Access the new design using the same table, record the data.


{| class="wikitable" style="text-align: center; margin: 0px auto;"
{| class="wikitable" style="text-align: center; margin: 0px auto;"
Line 255: Line 243:


The lab work is now complete. Please clean up the workstation. Return all unused materials to the TA. Refer to the [[#Assignment|Assignment]] section for the instructions needed to prepare the lab report.
The lab work is now complete. Please clean up the workstation. Return all unused materials to the TA. Refer to the [[#Assignment|Assignment]] section for the instructions needed to prepare the lab report.
= Assignment =
== Individual Lab Report ==
Follow the lab report guidelines laid out in the page called [[Specifications for Writing Your Lab Reports]]  in the Technical Communication section of this manual. The following  points should be addressed in the appropriate section of the lab report.
* Discuss product evaluation and the need for it
* Discuss quality improvement and its importance
* Explain why reverse engineering is important
* Discuss a standard, an average, accuracy, and precision
* Discuss the EV3 kit and software used
* Assess the techniques used in the testing process
* Discuss better testing methods for the robot design
* Describe how the components fit together and how the device was constructed
* Describe and show how the gear and velocity ratios were calculated, and discuss their importance in the  robot design
* Analyze the original robot design and the final design then  decide which design is better. Remember to substantiate the decision with the data, observations, and facts collected
* Make a final evaluation of the product: should it go into production? Describe its accuracy and precision
* Discuss the importance of ''%Prec'' and ''%Acc'' in determining product quality
* Recommend if any design improvements should be made
{{Lab notes}}
== Team PowerPoint Presentation ==
Follow the presentation guidelines laid out in the page called [[EG1004 Lab Presentation Format]] in the ''Introduction to Technical Presentations'' section of this manual. When preparing the presentation, consider the following points.
* Based on the test results, what is the product's marketability?
* If determinations were negative, what improvements could be made?
* Why is product evaluation important?
* Discuss why quality improvement is important
* Discuss the importance of gear and velocity ratios


= Footnotes =
= Footnotes =

Latest revision as of 02:10, 31 August 2022

Objectives

The objectives of this lab are to use product evaluation to determine if a robot design meets minimum standards for accuracy and precision, to use a quality improvement process to modify the robot design, to evaluate the modified robot design, and to use the data from the product evaluation of both robots will determine if the original robot or the modified robot should go into production.

Overview

A specified robot design will be assembled using an EV3 robotics kit. That design will be evaluated for its accuracy and precision in a distance test and an angle of deviation test. The results of that testing will be compared to a standard that will be set prior to testing. Using reverse engineering, the robot will be disassembled, analyzed, and quality improvements will be made to the design. The modified design will undergo the distance test and angle of deviation test. The data from the product evaluation of both robots will be used to determine if the original robot or the modified robot should go into production.

Product Evaluation

Product evaluation is a common task for an engineer and for companies. Sometimes the product evaluated will be a company's product, and sometimes companies will be evaluating a competitor's design. In either case, this evaluation will use a standard (an accepted value), determine an average, and calculate accuracy and precision.[1]

Professional engineers must be sure of standards. For the purpose of this lab, the EG1004 standard will be 80% precision and accuracy. This means that the product has to achieve a percent accuracy and percent precision of 80% and it has to pass 80% of the tests that it undergoes.[1]

When taking measurements, the first step is to determine an average (Equation 1).

Equation 1: Average.

In Equation 1, Ap is the average, P1, P2, P3, ..., Pn are the results of the tests, and N is the number of tests performed.

The average is compared to the standard. Accuracy is a variable that must always be considered. The degree of accuracy required will allow the results to be verified. Engineers often work on projects that affect people's safety so it is critical that the solutions to the problems solved by technical professionals are accurate.

The comparison of the average to the standard is called accuracy, defined as Acc in the following equation:

Equation 2: Accuracy.

In Equation 2, Ps is the standard value, and Ap is the value measured. Accuracy is always a positive number so use the absolute value in case Ps is less than Ap.

Precision is the repeatability of a result, or how close the results are to each other, defined as Prec in Equation 3:

Equation 3: Precision.

In Equation 3, Prec is precision, Phigh is the highest data value and Plow is the lowest data value. Like accuracy, precision is always a positive number so use an absolute value.

A product can be accurate but imprecise, or precise but inaccurate (Figure 1). The goal is to achieve a combination of the highest accuracy and precision. For engineers to make a recommendation based on experimental test data, it is important to determine how a product compares to a standard. Engineers need these benchmarks to determine the quality of a product.

Figure 1: Accuracy and precision.

By determining the percent accuracy of a product and comparing this percentage to a standard, the experimental evidence determines if the product has passed or failed (Equation 4).

Equation 4: Percent accuracy.

In Equation 4, %Acc is the percent accuracy, Ps is the standard value, and Ap is the value measured.

Percent accuracy shows how precise a product is compared to the standard (Equation 5).

Equation 5: Percent precision.

In Equation 5, %Prec is the percent precision, B is the exponential decay factor, and P is the precision measured. Each test requires the use of a different exponential decay factor (B):

Distance Test: B = 0.0972
Angle of Deviation Test: B = 0.0323

Calculating percent accuracy and percent precision and then comparing those results to a standard determines the quality of the product.

Quality Improvement

Quality improvement is a process of analyzing a design and testing it through physical modeling, computer simulation, or mathematical modeling. Quality improvement ensures that a product will perform as expected and allows improvements to be made to the design.

When deciding how to improve the design in this lab, keep these questions in mind:

  • Does the robot perform to standard?
    • If no, what can be done to improve its functionality? (Drive train, maneuverability, power output, programming, traction)
    • If yes, what can be done to improve its performance beyond the standard?

Changes will be implemented after testing the initial design.

Reverse Engineering

Reverse engineering software involves reversing a program's machine code to obtain the original source code or determine the file structures the program uses. This is done when source code is lost or is not available. For example, when a project is reactivated after being dormant for years to add new features to a product, the original design documentation may have been lost. Also, when companies are writing a program that must interface with a second company's software, the second company may not have any incentive to cooperate by describing how the data in the files they use are organized, forcing the first company to deduce it on their own. Sometimes, this process is also under¬taken as a way to improve the performance of a program, to fix a bug, or to find a virus. When the source code is obtained in this way for any of these reasons, it is legal and necessary. Reverse engineering software to copy it constitutes a copy¬right violation and is illegal.[2]

Hardware reverse engineering involves taking apart a device to see how it works. If a processor manufacturer wants to see how a competitor's processor works, the company can purchase the processor, disassemble it, and make a new processor similar to it. In some countries, this process is illegal. Hardware reverse engineering is quite expensive and requires an expert in the field.[2]

When beginning the process of reverse engineering the robot, identifying the gear trains and the individual gears used in the original design and in the final design is critical to understanding the robot's operation.

There are two types of gear trains: simple (Figure 2) and compound (Figure 3). A simple gear train has its gears arranged in a line (Figure 3).

Figure 2: Simple gear train.

Compound gear trains use axles to connect the component gears (see Figure 3).

Figure 3: Compound gear train.

Gear trains have measurable characteristics known as a gear ratio and a velocity ratio. These ratios are inversely proportional. Gear ratio equals output over input, and velocity ratio equals input over output. If multiple gears are being used, the gear ratio and velocity ratio are the product of these ratios for each individual gear. To determine input and output values, engineers count the number of teeth on each gear or by measure each gear's radius.

For example, to compute the gear ratio of the gear trains in Figures 2 and 3, these formulas (Figure 4) are used, where the values are the radii of the gears in the gear train.

Figure 4: Gear ratio.

Simple gear train

Compound gear train

For example, to compute the velocity ratio of the gear trains in Figures 2 and 3, these formulas (Figure 5) are used.

Figure 5: Velocity ratio.

Simple gear train

Compound gear train

It is important to note that the input is the source of rotation in a system, like a motor, and the output is the final gear of rotation in a system, like a wheel.

Figure 6: Types of Gears
Figures 6a-d courtesy of http://www.howthingswork.com
Figure 6f courtesy of Honda Motor Company, Inc.
Figure 6a: Rack gear. Its purpose is to change rotation into linear motion.
Figure 6b: Crown gear. Its purpose is to change the direction of rotation perpendicularly.
Figure 6c: Spur gear. Its purpose is to transmit torque through the gear train.
Figure 6d: Worm gear. Its purpose is to change the direction of rotation perpendicularly.
Figure 6e: Idler gear. Its purpose is to transfer force while not affecting the gear or velocity ratios.
Figure 6f: Pulley gear. Its purpose is to transmit rotation from one point to one or more other points.

The objective of reverse engineering the robot is to understand how it works and to calculate the gear and velocity ratios for its drive train (remember the wheels are part of the drive train). Remember that engineers reverse engineer products to reproduce or improve them. As the robot is disassembled, improvements to the robot's design should be considered.

Note: Using gear and velocity ratios have proven to be extremely helpful when building robots for the EG1004 Semester-Long Design Project.

Design Considerations

Carefully consider the lab objectives and what needs to be done to achieve them:

  • What made or will make the results inaccurate and imprecise?
  • If the robot failed a test, why did it fail?

Materials and Equipment

  • A Mindstorms kit
  • A computer with Mindstorms software
  • A crotractor and a ruler
  • Graph paper

Procedure

Problem Statement

Evaluate a robot design by building a specified robot design. Test the robot design, make improvements on the design, and retest it. While taking it apart, analyze the drive train.

  1. Build a robot according to the design assigned by the TA
  2. Start the Mindstorms EV3 software
  3. Click on the + in the top left corner to open up a blank program
    Figure 6: Mindstorms start page.
  4. The screen should look like the screen shot in Figure 6. The program begins with the Start block (Figure 7)
    Figure 7: Start block.
  5. 5. Place a Large Motor block from the Action palette next to the Start block. Along the bottom of the Large Motor block, there are four options (Figure 8). On the left, there is a Mode Selector button followed by three values for the input parameters picture above the value: from left to right, Power, Rotations, and Brake at End
    Figure 8: Motor block.
  6. Click on the Mode Selector (Figure 9) and change the mode of the Large Motor block to On for Seconds. This will change the Rotations input parameter to Seconds
    Figure 9: Mode selector.
  7. Set Power to 100 and Seconds to five (Figure 10) . Using these values, the motor will move at 100% power for five seconds and then brake to a stop
    Figure 10
  8. 8. Assign the port that the motor is connected to at the top right corner of the block. Assume the motor is connected to Port A (Figure 11)
    Figure 11: Port assignment.
  9. 9. Place a Wait block from the Flow Control palette after the Large Motor block. By default, the Wait block will wait for one second. Change the Seconds input parameter value to three so that there is a three second pause in the execution of the program (Figure 12)
    Figure 12: Wait block.
  10. Select an Invert Motor block from the Advanced palette and position it after the Wait block. By default, the Invert parameter is set to True (Figure 13). Following the execution of this block, motors will move in reverse until another Invert Motor block is executed with the Invert parameter set to False
    Figure 13: Invert parameter.
  11. Take another Large Motor block from the Action palette and place it after the Invert Motor block. Set Mode to On for Seconds, Power to 100, Seconds to five, and Port to A (Figure 14)
    Figure 14: Completed program.
  12. Run the program. The program should run the motor connected to Port A for five seconds at 100% power, wait for three seconds, then run the motor connected to Port A in reverse for five seconds at 100% power.

Program and test the robot for each test specified. To do this, make a list of the tests. Brainstorm what the robot should do in each test. Then, write the program. Record the standard, average, accuracy, and precision. If the robot does not operate correctly, troubleshoot the problem. The TA must initial the original data.

Note: The initial program cannot be used for the following tests.

Distance Test

Note: Use centimeters for this test.

In this test, the robot will go forward a set distance in a specified time. Test the robot's accuracy and precision for distance. Before testing begins, the standard must be found. To find the standard, program the robot to go forward for five seconds and measure the distance it travels. Divide the result by five, and then multiply by four. This result is the standard for distance.

Equation 6: Distance test, standard formula.

Once the standard has been obtained, the testing can begin.

Figure 15: Sample drawing paper with data. The green line represents the five-second standard measurement. The blue lines represent the five four-second trials.

Warning: Read the section on the Angle of Deviation Test before performing this test.

  1. Upload a program that will prompt the robot to travel for four seconds
  2. Measure the distance traveled
  3. Repeat this test five times

Angle of Deviation Test

Note: This test is performed at the same time as the distance test. Since the robot is expected to travel in a straight line, the standard is 360°.

Figure 16: Standard for the angle of deviation test.

In this test, the robot will move forward for four seconds and the angle of deviation will be recorded. The angle of deviation is the angle between a center line or a 0º line and a line drawn from the starting point to where the robot finishes.

  1. Use a protractor to draw a 0° line from the start point
  2. Mark the start point and place the robot there. Identify a reference point on the robot
  3. Run the distance test
  4. Make a mark at the point where the robot finished, and draw a line from the start point to this mark
  5. Measure the positive angle of deviation
  6. Repeat this procedure five times


Quality Improvement

  1. Assess the robot design using the data collected and from observations
  2. Make necessary adjustments
  3. Retest the robot and collect data
  4. Take pictures of both the original design and the modified design

Reverse Engineering

  1. Visually assess the robot's design. Consider how it works
  2. Sketch the front, top, and most detailed side of the robot. Be sure to include dimensions in the sketch
  3. Also sketch the most detailed view of the gear train
  4. Before disassembling the robot, ask the TA to take a picture of it
  5. Disassemble the robot. Analyze its inner workings and sketch the gear train(s). Note: All sketches in EG1004 must be done in pencil
  6. Have all sketches and original data signed by the TA

With measurements for both tests performed and an analysis of the drive train of the robot, begin data analysis. Calculate the average, accuracy, and precision for each test. Compare the robots’ percent accuracy and percent precision to the standard to determine if the robot is accurate and precise. Label all major components and describe their functions. Calculate percent accuracy and percent precision. Calculate the gear and velocity ratios.

Examples
  • If Ps was five cm and Ap was seven cm, then the percent accuracy (%Acc) is 60%. This test does not pass the 80% standard
  • If Ps was five cm and Ap was 25cm, then the percent accuracy is -300%. Since a negative percentage has no meaning in engineering, it can be said that the product has a percent accuracy of zero. Any percent accuracy less than zero is equal to zero
  • If Ps was 10cm and Ap was eight cm, then the percent accuracy is 80%, which is a passing result
  • For the distance test, if the precision (P) was 0.5cm, the percent precision (%Prec) is 95.3%. This robot passes the 80% standard.
  • For the angle of deviation test, if the precision (P) was 10.25°, the percent precision is 71.8%. This robot does not pass the 80% standard.

Tabulation of Results

To decide if the product is ready to manufacture, arrange the data in a table designed to calculate its precision and accuracy. Remember it must pass all the tests by at least 80%.

Original Design Accuracy %Acc Pass/Fail Precision %Prec Pass/Fail
Distance Test
Angle of Deviation Test

Access the new design using the same table, record the data.

New Design Accuracy %Acc Pass/Fail Precision %Prec Pass/Fail
Distance Test
Angle of Deviation Test

The lab work is now complete. Please clean up the workstation. Return all unused materials to the TA. Refer to the Assignment section for the instructions needed to prepare the lab report.

Assignment

Individual Lab Report

Follow the lab report guidelines laid out in the page called Specifications for Writing Your Lab Reports in the Technical Communication section of this manual. The following points should be addressed in the appropriate section of the lab report.

  • Discuss product evaluation and the need for it
  • Discuss quality improvement and its importance
  • Explain why reverse engineering is important
  • Discuss a standard, an average, accuracy, and precision
  • Discuss the EV3 kit and software used
  • Assess the techniques used in the testing process
  • Discuss better testing methods for the robot design
  • Describe how the components fit together and how the device was constructed
  • Describe and show how the gear and velocity ratios were calculated, and discuss their importance in the robot design
  • Analyze the original robot design and the final design then decide which design is better. Remember to substantiate the decision with the data, observations, and facts collected
  • Make a final evaluation of the product: should it go into production? Describe its accuracy and precision
  • Discuss the importance of %Prec and %Acc in determining product quality
  • Recommend if any design improvements should be made

Remember: Lab notes must be taken. Experimental details are easily forgotten unless written down. EG1004 Lab Notes Paper can be downloaded and printed from the EG1004 Website. Use the lab notes to write the Procedure section of the lab report. At the end of each lab, a TA will scan the lab notes and upload them to the Lab Documents section of the EG1004 Website. One point of extra credit is awarded if the lab notes are attached at the end of the lab report. Keeping careful notes is an essential component of all scientific practice.

Team PowerPoint Presentation

Follow the presentation guidelines laid out in the page called EG1004 Lab Presentation Format in the Introduction to Technical Presentations section of this manual. When preparing the presentation, consider the following points.

  • Based on the test results, what is the product's marketability?
  • If determinations were negative, what improvements could be made?
  • Why is product evaluation important?
  • Discuss why quality improvement is important
  • Discuss the importance of gear and velocity ratios

Footnotes

  1. ^ a b Oakes, W.C., L.L. Leone, and C.G. Gunn, Engineering Your Future, MI: Great Lakes Press, 2002.
  2. ^ a b What Is website. TechTarget Network. Retrieved July 29th, 2003.