Difference between revisions of "Hardware Analysis and Synthesis"
Line 1: | Line 1: | ||
<h1 align | <html> | ||
<style> | |||
h1 {color:#4444cc; text-align:center} | |||
h2 {color:#4444cc} | |||
p.caption {font-family:Arial; text-align:center; font-weight:bold} | |||
</style> | |||
<head> | |||
<title>EG1004 Lab 3: Product Evaluation</title> | |||
</head> | |||
<h1>EG1004 Lab 3: Product Evaluation</h1> | |||
<h2>1 OBJECTIVES</h2> | <h2>1 OBJECTIVES</h2> | ||
<p>Together, | <p>The experimental goal of this lab is for your team to determine if a robot | ||
we will define the concept of product evaluation, demonstrate the importance of | kit meets minimum standards for accuracy and precision. To do this your team | ||
accurate measurements, and show how they relate to both precision and accuracy. | will build and program a robot, and test it against the provided EG1004 | ||
Once you have completed your lab work, you will be able to describe the method | standard. All your results must be tabulated accurately. </p> | ||
of troubleshooting you developed.</p> | |||
<p>Together, we will define the concept of product evaluation, demonstrate the | |||
importance of accurate measurements, and show how they relate to both | |||
precision and accuracy. Once you have completed your lab work, you will be able | |||
to describe the method of troubleshooting you developed.</p> | |||
<h2>2 OVERVIEW</h2> | <h2>2 OVERVIEW</h2> | ||
<p>Your team has been hired by an engineering firm to evaluate | <p>Your team has been hired by an engineering firm to evaluate a robotics kit. | ||
a robotics kit. This Robotics kit has a manufacturer’s suggested retail price | This Robotics kit has a manufacturer’s suggested retail price of $249.99. The | ||
of $ | firm has specified the robot design they would like you to build and test. | ||
build and test. Your assignment is to test the kit and the specified design, | Your assignment is to test the kit and the specified design, describe what works | ||
describe what works and list the improvements that should be made to both the | and list the improvements that should be made to both the kit and the robot | ||
kit and the robot design you built. The price of the kit should be factored | design you built. The price of the kit should be factored into the overall | ||
into the overall product evaluation. What other factors do engineers consider | product evaluation. What other factors do engineers consider when they | ||
when they evaluate products?</p> | evaluate products?</p> | ||
<p>Product evaluation is a common task for an engineer. Sometimes the product you | <p>Product evaluation is a common task for an engineer. Sometimes the product | ||
evaluate will be one your team built, and sometimes you will be analyzing a | you evaluate will be one your team built, and sometimes you will be analyzing | ||
competitor's design. In either case, you will need to decide on a <b><i>standard</i></b> | a competitor's design. In either case, you will need to decide on a <b><i> | ||
an accepted value), determine an <b><i>average</i></b>, and calculate <b><i>accuracy</i> | standard</i></b> an accepted value), determine an <b><i>average</i></b>, and | ||
and <b><i>precision</i></b>.<sup>1</sup></p> | calculate <b><i>accuracy</i> and <b><i>precision</i></b>.<sup>1</sup></p> | ||
<p>Professional engineers must be sure of standards. For the purpose of this lab, | <p>Professional engineers must be sure of standards. For the purpose of this | ||
the EG1004 Standard that you will be testing to is 80% precision and accuracy. This | lab, the EG1004 Standard that you will be testing to is 80% precision and | ||
means that the product you will be testing has to pass 80% of the tests that it | accuracy. This means that the product you will be testing has to pass 80% of the | ||
undergoes. Some projects may require a degree of accuracy to a tenth of an inch. | tests that it undergoes. Some projects may require a degree of accuracy to a | ||
Attempting to make the project accurate to the hundredth might be unnecessary and | tenth of an inch. Attempting to make the project accurate to the hundredth might | ||
difficult to control. It is up to you to verify that your solutions are accurate | be unnecessary and difficult to control. It is up to you to verify that your | ||
based on the standard for your design project.<sup> | solutions are accurate based on the standard for your design project.<sup> <p> | ||
When taking measurements, the first step is to determine an average. </p> | |||
<p align=center>[[image:lab_hardsyn_1.gif]]</p> | <p align=center>[[image:lab_hardsyn_1.gif]]</p> | ||
<p>In this equation, <i>A<sub>P</sub></i> is the average, <i>P<sub>1</sub></i>, <i>P<sub>2</sub></i>, <i>P<sub>3</sub></i>, … | <p>In this equation, <i>A<sub>P</sub></i> is the average, <i>P<sub>1</sub></i>, | ||
results of the tests and <i>N</i> is the total number of tests to be performed.</p> | <i>P<sub>2</sub></i>, <i>P<sub>3</sub></i>, … are the results of the tests and | ||
<i>N</i> is the total number of tests to be performed.</p> | |||
<p>The <b><i>average</i></b> is then compared to the <b><i>standard</i></b>. | <p>The <b><i>average</i></b> is then compared to the <b><i>standard</i></b>. | ||
Accuracy is a variable you must always consider. The degree of accuracy | Accuracy is a variable you must always consider. The degree of accuracy required | ||
will allow you to verify your results. Engineers often work on projects that | |||
projects that affect people's safety. Therefore, it is critical that the | affect people's safety. Therefore, it is critical that the solutions to the | ||
solutions to the problems you will solve as technical professionals are <b><i>accurate</i>< | problems you will solve as technical professionals are <b><i>accurate</i> <p>The | ||
comparison of the average to the standard is called accuracy, defined as Acc in | |||
the following equation: </p> | |||
<p align=center><i>Acc = |P<sub>S</sub> - A<sub>P</sub>|</i></p> | <p align=center><i>Acc = |P<sub>S</sub> - A<sub>P</sub>|</i></p> | ||
<p>In this equation, P<sub>S</sub> is the standard value, and A<sub>P</sub> is the value you measured. | <p>In this equation, P<sub>S</sub> is the standard value, and A<sub>P</sub> is | ||
Accuracy is always a positive number, so we take the absolute value in case P<sub>S</sub> is less than | the value you measured. Accuracy is always a positive number, so we take the | ||
A<sub>P</sub></p> | absolute value in case P<sub>S</sub> is less than A<sub>P</sub></p> | ||
<p>Precision is the repeatability of a result, or how close the results are to each other, defined as | <p>Precision is the repeatability of a result, or how close the results are to | ||
Prec in the following equation:</p> | each other, defined as Prec in the following equation:</p> | ||
<p align=center><i>Prec = |P<sub>high</sub> - P<sub>low</sub>|</i></p> | <p align=center><i>Prec</i><i> = |P<sub>high</sub> - P<sub>low</sub>|</i></p> | ||
<p>In this equation, <i>Prec</i> is | <p>In this equation, <i>Prec</i> is Precision, <i>P<sub>high</sub></i> is the | ||
and <i>P<sub | highest data value and <i>P<sub> is the lowest data value. Like accuracy, | ||
we use an absolute value here as well.</p> | precision is always a positive number, so we use an absolute value here as | ||
well.</p> | |||
<p>A product test can be accurate but imprecise, or precise but | <p>A product test can be accurate but imprecise, or precise but inaccurate. | ||
inaccurate. The goal is always the combination of accuracy and precision. In | The goal is always the combination of accuracy and precision. In order for | ||
order for engineers to make a recommendation based on experimental test data, | engineers to make a recommendation based on experimental test data, it is | ||
it is important to determine how a device measures against a standard. | important to determine how a device measures against a standard. Engineers | ||
need these benchmarks to determine the quality of a product.</p> | |||
<p align=center>[[image:lab_hardsyn_12.gif]]</p> | <p align=center>[[image:lab_hardsyn_12.gif]]</p> | ||
<p>By determining the percent accuracy of an experiment, and comparing this | <p>By determining the percent accuracy of an experiment, and comparing this | ||
percentage to a standard, you can determine, based on experimental evidence, whether | percentage to a standard, you can determine, based on experimental evidence, | ||
the prototype has passed or failed. The equation for percent accuracy is: </p> | whether the prototype has passed or failed. The equation for percent accuracy | ||
is: </p> | |||
<p align=center>[[image:lab_hardsyn_13.gif]]</p> | <p align=center>[[image:lab_hardsyn_13.gif]]</p> | ||
<p>In this equation, <i>%Acc</i> is the percent accuracy, <i>P<sub>S</sub></i> is the standard | <p>In this equation, <i>%Acc</i> is the percent accuracy, <i>P<sub>S</sub></i> | ||
value, and <i>A<sub>P</sub></i> is the value you measured.</p> | is the standard value, and <i>A<sub>P</sub></i> is the value you measured.</p> | ||
<p>Percent accuracy allows an engineer to determine how precise a device is compared to | <p>Percent accuracy allows an engineer to determine how precise a device is | ||
the standard. The equation for percent precision is: </p> | compared to the standard. The equation for percent precision is: </p> | ||
<p align=center><i>%Prec = 100e<sup>-BP</sup></i></p> | <p align=center><i>%Prec = 100e<sup>-BP</sup></i></p> | ||
<p>In this equation, <i>%Prec</i> is the percent precision, < | <p>In this equation, <i>%Prec</i> is the percent precision, <i>B</i> is the | ||
and <i>P</i> | exponential decay factor, and <i>P</i> precision you measured.</p> | ||
<p>Calculating percent accuracy and percent precision and then | <p>Calculating percent accuracy and percent precision and then comparing your | ||
comparing your results to a standard will allow you to determine the quality of | results to a standard will allow you to determine the quality of your | ||
prototype.</p> | |||
<h2>3 YOUR ASSIGNMENT</h2> | <h2>3 YOUR ASSIGNMENT</h2> | ||
Line 106: | Line 119: | ||
addressed in the appropriate section of your lab report:</p> | addressed in the appropriate section of your lab report:</p> | ||
<ul> | <ul type=disc> | ||
<li>Discuss product evaluation and the importance of testing.</li> | <li>Discuss product evaluation | ||
<li>Explain the importance of designing to a standard.</li> | and the importance of testing.</li> | ||
<li>Discuss average, accuracy, and precision.</li> | <li>Explain the importance of | ||
<li>Assess the techniques used in the testing process.</li> | designing to a standard.</li> | ||
<li>Discuss better testing methods for the robot design.</li> | <li>Discuss average, accuracy, | ||
<li>Make a final evaluation of the product. Describe its accuracy and precision.</li> | and precision.</li> | ||
<li>Discuss the importance of % precision and % accuracy in determining product quality.</li> | <li>Assess the techniques used in | ||
<li>State your recommendations.</li> | the testing process.</li> | ||
<li>Discuss better testing | |||
methods for the robot design.</li> | |||
<li>Make a final evaluation of | |||
the product. Describe its accuracy and precision.</li> | |||
<li>Discuss the importance of % | |||
precision and % accuracy in determining product quality.</li> | |||
<li>State your recommendations.</li> | |||
</ul> | </ul> | ||
<h3>Team PowerPoint Presentation</h3> | <h3>Team PowerPoint Presentation</h3> | ||
Line 123: | Line 142: | ||
section of this manual. When you are preparing your presentation, consider the following points:</p> | section of this manual. When you are preparing your presentation, consider the following points:</p> | ||
<ul> | <ul type=disc> | ||
<li>Based on the test results, what is your determination for the product's marketability?</li> | <li>Based on the test results, | ||
<li>If determinations were negative, what improvements could be made?</li> | what is your determination for the product's marketability?</li> | ||
<li>Why is product evaluation important to you?</li> | <li>If determinations were negative, | ||
what improvements could be made?</li> | |||
<li>Why is product evaluation | |||
important to you?</li> | |||
</ul> | </ul> | ||
<h2>4 MATERIALS AND EQUIPMENT</h2> | <h2>4 MATERIALS AND EQUIPMENT</h2> | ||
<ul> | <ul type=disc> | ||
<li> | <li>Minstorms | ||
<li>Computer with Mindstorms Software</li> | Kit</li> | ||
<li>Protractor and Ruler</li> | <li>Computer with Mindstorms Software</li> | ||
<li>Protractor and Ruler</li> | |||
</ul> | </ul> | ||
Line 149: | Line 171: | ||
<h3>Problem Statement</h3> | <h3>Problem Statement</h3> | ||
<p>You have been hired by an engineering firm to build and | |||
evaluate a robotics kit by building a specified robot design. You have been | <p>You have been hired by an engineering firm to build and evaluate a robotics | ||
asked to test the robot design and the overall kit, and to make suggestions for | kit by building a specified robot design. You have been asked to test the | ||
robot design and the overall kit, and to make suggestions for design | |||
improvements. </p> | |||
<ol> | <ol> | ||
<li>Build a robot according to the lab handout provided by your TA.</li> | <li>Build a robot according to the lab handout provided by your TA.</li> | ||
<p><b | <p><b>Warning: Do <i>not</i> disassemble the robot when you finish. You will use | ||
finish. You will use it again in the next lab.</b></p> | it again in the next lab.</b></p> | ||
<li>Create a program in Mindstorms: | <li>Create a program in Mindstorms: | ||
<ol type= | <ol type=a> | ||
< | <li>Start the Mindstorms Program. On the welcome screen type in a name for your | ||
program and press Go.</li> | |||
< | <p align=center>[[image:lab_hardsyn_3.jpg]]</p> | ||
<p> | <p>b. Click on Complete Pallet to see all the programming icons.</p> | ||
<p> | <p> Turn off Robot Educator by clicking the small X.</p> | ||
<p> | <p><img src="lab3_files/image005.jpg"></p> | ||
<p> | <p> </p> | ||
< | <p> </p> | ||
< | |||
<p>c. Your screen should look like this. You are now ready to write a | |||
program.</p> | |||
< | <p><img src="lab3_files/image006.jpg"></p> | ||
< | <p> </p> | ||
<p> | <p> </p> | ||
<p> | <p>d. Click on Actions, then on the Motor icon to select it. </p> | ||
<p> | <p><img src="lab3_files/image007.jpg"></p> | ||
< | <p> </p> | ||
< | <p> </p> | ||
< | <p>e. Place the motor icon on to the “start�? square.</p> | ||
< | <p><img src="lab3_files/image008.jpg"></p> | ||
< | <p> </p> | ||
< | <p> </p> | ||
< | <p>f. Click the motor icon to see its properties. Here you can define exactly | ||
how you want the motor to spin and when to stop.</p> | |||
< | <p><img src="lab3_files/image009.jpg"></p> | ||
</ | |||
< | <p> </p> | ||
<p> </p> | <p> </p> | ||
< | <p>g. Continue writing your program by placing icons after each other. The | ||
execution of your program will run from left to right. </p> | |||
< | <p> In the sample program below, motor A will spin forward for 5 seconds, | ||
Stop, Wait 5 seconds, and then spin backwards for another 5 seconds.</p> | |||
<p> | <p><img src="lab3_files/image010.jpg"></p> | ||
<p> | <p> </p> | ||
<p> </p> | <p> </p> | ||
< | <p>h. For this lab you can use the Move icon to easily instruct your robot to | ||
move.</p> | |||
<p> | <p><img src="lab3_files/image011.jpg"></p> | ||
<p> </p> | <p> </p> | ||
<p> </p> | <p> </p> | ||
< | <p>i. Whenever you need to, you can | ||
use Robot Educator to quickly learn how to program in Lego Mindstorms.</p> | |||
<p> | <p><img src="lab3_files/image012.jpg"></p> | ||
<p> | <p> </p> | ||
<p> | <p> </p> | ||
<p><b>Note:</b> Program and test your robot for each test specified. | <p><b>Note:</b> Program and test your robot for each test specified. To do this, | ||
To do this, make a list of the tests in your lab notebook. Brainstorm what you | make a list of the tests in your lab notebook. Brainstorm what you want the | ||
want the robot to do in each test. Then, use your notes to write your program. | robot to do in each test. Then, use your notes to write your program. Record the | ||
Record the standard, average, accuracy, and precision. If your robot does not | standard, average, accuracy, and precision. If your robot does not operate | ||
correctly, troubleshoot the problem. Your TA must initial your original | |||
data.</p> | data.</p> | ||
<h3>Distance Test</h3> | <h3>Distance Test</h3> | ||
<p>< | <p><b><i>Note:</i></b><i> Use the Metric system for this test</i></p> | ||
< | |||
< | <p>In this test, the robot will go forward a set distance in a specified time. | ||
You are to test the robot's accuracy and precision for this distance. Before | |||
testing begins the standard must be found. To find the standard, program the | |||
robot to go forward for 5 seconds and measure the distance it travels. Divide | |||
your result by 5, and then multiply by 4. This number is your standard. Once the | |||
standard has been obtained, the testing can begin.</p> | |||
<p>< | <p><b>Warning: Read the section on the Angle of Deviation Test before performing | ||
Since the robot is expected | this test.</b> <ol start=1 type=1> <li>Upload a program that will set the | ||
robot to travel for 4 seconds</li> <li>Measure the distance traveled. | |||
</li> <li>Repeat this test 10 times.</li> </ol> <h3>Angle of Deviation | |||
Test</h3> | |||
<p><b><i>Note:</i></b><i> This test can be performed at the same time | |||
as the Distance Test. Since the robot is expected to travel in a straight line, | |||
the standard is 360°.</i></p> | |||
<p><i>In this test, the | <p><i>In this test, the robot will move forward for 4 seconds and the angle of | ||
robot will move forward for | deviation will be recorded. The angle of deviation is the angle between a | ||
recorded. The angle of deviation is the angle between a straight line and a | straight line and a line drawn from the starting point to where the robot | ||
line drawn from the starting point to where the robot finishes.</i></p> | finishes.</i></p> | ||
<ol> | <ol start=1 type=1> | ||
<li>Use a protractor to draw a 0° line from the start point.</li> | <li>Use a protractor to draw a 0° | ||
line from the start point.</li> | |||
<li>Mark the start point and place your robot there. Identify a reference point on the robot.</li> | <li>Mark the start point and | ||
place your robot there. Identify a reference point on the robot.</li> | |||
<li>Run the Distance Test.</li> | <li>Run the Distance Test.</li> | ||
<li>Make a mark at the point | |||
<li>Make a mark at the point where the robot finished, and | where the robot finished, and draw a line from the start point to this | ||
draw a line from the start point to this mark.</li> | mark.</li> | ||
<li>Measure the positive angle of | |||
<li>Measure the positive angle of deviation.</li> | deviation.</li> | ||
<li>Repeat this procedure 10 | |||
<li>Repeat this procedure 10 times.</li> | times.</li> | ||
</ol> | </ol> | ||
<h3>Rotation Test</h3> | <h3>Rotation Test</h3> | ||
<p>In this test, the robot will rotate around an axis. Again, | <p>In this test, the robot will rotate around an axis. Again, the standard | ||
the standard must be obtained. By running the rotation test for 5 seconds and | must be obtained. By running the rotation test for 5 seconds and dividing the | ||
dividing the angle obtained by 5, you will determine how many degrees your | angle obtained by 5, you will determine how many degrees your robot rotates | ||
robot rotates per second. This is the standard. With the standard in mind, | per second. This is the standard. With the standard in mind, project how many | ||
project how many degrees your robot will rotate in 4 seconds.</p | degrees your robot will rotate in 4 seconds.</p> | ||
<ol start=1 type=1> | |||
<li>Write the program in Mindstorms to perform this test. Your TA will | |||
explain | |||
this process.</li> | |||
<li>Upload your rotation program | |||
from the computer to the NXT.</li> | |||
<li>Make a center point on a | |||
sheet of paper.</li> | |||
<li>Run the program for 5 | |||
seconds.</li> | |||
<li>Mark the reference point | |||
after rotation and measure the rotation angle.</li> | |||
<li>Divide this value by 5 to | |||
determine the angle of rotation for 1 second.</li> | |||
<li>Multiply the angle of | |||
rotation per second by 4 to obtain the standard.</li> | |||
<li>Go back to the Mindstorms program and change the time from 5 seconds | |||
to 4 seconds.</li> | |||
<li>Upload the new program to the | |||
NXT.</li> | |||
<li>Run the program 10 times. | |||
Make sure to measure the angle the robot rotated by each time.</li> | |||
</ol> | </ol> | ||
<p>You now have measurements for each test you have performed. You must | |||
analyze this data. Calculate the average, accuracy, and precision for each | |||
test. Compare your results to the standard in order to determine if your robot | |||
is accurate and precise. You must also calculate percent accuracy and percent | |||
precision. Use this formula to calculate percent accuracy:</p> | |||
<p> | <p align=center><a | ||
href="https://egmanual.poly.edu/index.php?title=Image:Lab_hardsyn_13.gif" | |||
title="image:lab_hardsyn_13.gif"><img src="lab3_files/image003.gif" ></a></p> | |||
<p>In this equation, <i>%Acc</i> is the percent accuracy, <i>P<sub>S</sub></i> is the standard | <p>In this equation, <i>%Acc</i> is the percent accuracy, <i>P<sub>S</sub></i> | ||
value, and <i>A<sub>P</sub></i> is the value you measured.</p> | is the standard value, and <i>A<sub>P</sub></i> is the value you measured.</p> | ||
<h3>Examples:</h3> | <h3>Examples:</h3> | ||
<ol> | <ol start=1 type=1> | ||
<li>If <i>P<sub>S</sub></i> was determined to be 5cm and <i>A<sub>P</sub></i> was found to be 7cm, | <li>If <i>P<sub>S</sub></i> was | ||
determined to be 5cm and <i>A<sub>P</sub></i> was found to be 7cm, then | |||
the percent accuracy (<i>%Acc</i>) is 60%. This test does not pass the | |||
<li>If <i>P<sub>S</sub></i> was determined to be 5 feet and <i>A<sub>P</sub></i> was found to be 25 | 80% | ||
feet, then the result of the equation is -300%. Since a negative percentage has | criterion.</li> | ||
no meaning in engineering, it can be said that the product has a percent accuracy | <li>If <i>P<sub>S</sub></i> was | ||
of zero. Any percent accuracy less than zero is considered to be equal to zero.</li> | determined to be 5 feet and <i>A<sub>P</sub></i> was found to be 25 feet, | ||
then the result of the equation is -300%. Since a negative percentage has | |||
<li>If <i>P<sub>S</sub></i> was determined to be 10 ft, and <i>A<sub>P</sub></i> was found to be | no meaning in engineering, it can be said that the product has a percent | ||
8 ft, then the result of the equation is exactly 80%, which is a passing result.</li> | accuracy of zero. Any percent accuracy less than zero is | ||
considered to be equal to zero.</li> | |||
<li>If <i>P<sub>S</sub></i> was | |||
determined to be 10 ft, and <i>A<sub>P</sub></i> was found to be 8 ft, | |||
then | |||
the result of the equation is exactly 80%, which is a passing result.</li> | |||
</ol> | </ol> | ||
<p>Use this formula to calculate % precision: </p> | <p>Use this formula to calculate % precision: </p> | ||
<p align=center><i>%Prec = 100e<sup>-BP</sup></i></p> | <p align=center><i>%Prec = 100e<sup>-BP</sup></i></p> | ||
<p>In this equation, <i>%Prec</i> is the percent precision, <i>B</i> is the exponential | <p>In this equation, <i>%Prec</i> is the percent precision, <i>B</i> is the | ||
decay factor, and <i>P</i> | exponential decay factor, and <i>P</i> precision you measured. Each individual | ||
requires the use of a different exponential decay factor (<i>B</i>):</p> | test requires the use of a different exponential decay factor (<i>B</i>):</p> | ||
<p align=center><i>Distance Test: B = 0.0972</i></p> | <p align=center><i>Distance Test: B = 0.0972</i></p> | ||
Line 419: | Line 393: | ||
<h3>Examples:</h3> | <h3>Examples:</h3> | ||
<ol> | <ol start=1 type=1> | ||
<li>For the Distance Test, if the precision <i>(P)</i> was measured | <li>For the Distance Test, if the | ||
to be 0.5 cm, the percent precision <i>(%Prec)</i> is 95.3%. This robot | precision <i>(P)</i> was measured to be 0.5 cm, the percent precision <i> | ||
passes the 80% criterion for this test.</li> | (%Prec)</i> is 95.3%. This robot passes the 80% | ||
criterion for this test.</li> | |||
<li>For the Angle of Deviation Test, if the precision <i>(P)</i> was | <li>For the Angle of Deviation | ||
measured to be 10.25°, the percent precision <i>(%Prec)</i> | Test, if the precision <i>(P)</i> was measured to be 10.25°, the percent | ||
This robot does not pass the 80% criterion.</li> | precision <i>(%Prec)</i> is 71.8%. This robot | ||
does not pass the 80% criterion.</li> | |||
</ol> | </ol> | ||
<h3>Tabulation of Results</h3> | <h3>Tabulation of Results</h3> | ||
<p>To decide if the product is ready to manufacture, arrange | <p>To decide if the product is ready to manufacture, arrange your data in a | ||
your data in a table designed to calculate its precision and accuracy. Remember | table designed to calculate its precision and accuracy. Remember it must pass | ||
it must pass all the tests at a rate of at least 80%.</p> | all the tests at a rate of at least 80%.</p> | ||
<table border=1 cellspacing=0 | <table border=1 cellspacing=0> | ||
<tr> | <tr> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p>Accuracy</p> | <p>Accuracy</p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p>%Acc</p> | <p>%Acc</p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p>Pass/Fail</p> | <p>Pass/Fail</p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p>Precision </p> | <p>Precision </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p>%Prec</p> | <p>%Prec</p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p>Pass/Fail</p> | <p>Pass/Fail</p> | ||
</td> | </td> | ||
</tr> | </tr> | ||
<tr> | <tr> | ||
<td> | <td> | ||
<p>Distance Test</p> | <p>Distance Test</p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
Line 475: | Line 455: | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
</tr> | </tr> | ||
Line 490: | Line 473: | ||
<td> | <td> | ||
<p>Angle of Deviation Test</p> | <p>Angle of Deviation Test</p> | ||
</td> | </td> | ||
<td> | <td> | ||
Line 497: | Line 481: | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
</tr> | </tr> | ||
Line 519: | Line 507: | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
<td> | <td> | ||
<p> </p> | <p> </p> | ||
</td> | </td> | ||
</tr> | </tr> | ||
</table> | </table> | ||
<p>Your lab work is now complete. Please clean up your workstation. Return all | |||
unused materials to your TA. Refer to section <b><i>3 Your Assignment</i></b> | |||
for the instructions you need to prepare your lab report.</p> | |||
< | <h2>Footnotes</h2> | ||
< | <p><sup>1</sup> Oakes, W.C., L.L. Leone, and C.G. Gunn, <i>Engineering Your | ||
Future</i> w:st="on">MI: Great Lakes Press, 2002</p> | |||
<p> | <p> </p> | ||
</html> |
Revision as of 21:50, 12 September 2007
<html> <style> h1 {color:#4444cc; text-align:center} h2 {color:#4444cc} p.caption {font-family:Arial; text-align:center; font-weight:bold} </style>
<head> <title>EG1004 Lab 3: Product Evaluation</title> </head>
EG1004 Lab 3: Product Evaluation
1 OBJECTIVES
The experimental goal of this lab is for your team to determine if a robot kit meets minimum standards for accuracy and precision. To do this your team will build and program a robot, and test it against the provided EG1004 standard. All your results must be tabulated accurately.
Together, we will define the concept of product evaluation, demonstrate the importance of accurate measurements, and show how they relate to both precision and accuracy. Once you have completed your lab work, you will be able to describe the method of troubleshooting you developed.
2 OVERVIEW
Your team has been hired by an engineering firm to evaluate a robotics kit. This Robotics kit has a manufacturer’s suggested retail price of $249.99. The firm has specified the robot design they would like you to build and test. Your assignment is to test the kit and the specified design, describe what works and list the improvements that should be made to both the kit and the robot design you built. The price of the kit should be factored into the overall product evaluation. What other factors do engineers consider when they evaluate products?
Product evaluation is a common task for an engineer. Sometimes the product you evaluate will be one your team built, and sometimes you will be analyzing a competitor's design. In either case, you will need to decide on a standard an accepted value), determine an average, and calculate accuracy and precision.1
Professional engineers must be sure of standards. For the purpose of this lab, the EG1004 Standard that you will be testing to is 80% precision and accuracy. This means that the product you will be testing has to pass 80% of the tests that it undergoes. Some projects may require a degree of accuracy to a tenth of an inch. Attempting to make the project accurate to the hundredth might be unnecessary and difficult to control. It is up to you to verify that your solutions are accurate based on the standard for your design project.
When taking measurements, the first step is to determine an average.
In this equation, AP is the average, P1, P2, P3, … are the results of the tests and N is the total number of tests to be performed.
The average is then compared to the standard. Accuracy is a variable you must always consider. The degree of accuracy required will allow you to verify your results. Engineers often work on projects that affect people's safety. Therefore, it is critical that the solutions to the problems you will solve as technical professionals are accurate
The comparison of the average to the standard is called accuracy, defined as Acc in the following equation:
Acc = |PS - AP|
In this equation, PS is the standard value, and AP is the value you measured. Accuracy is always a positive number, so we take the absolute value in case PS is less than AP
Precision is the repeatability of a result, or how close the results are to each other, defined as Prec in the following equation:
Prec = |Phigh - Plow|
In this equation, Prec is Precision, Phigh is the highest data value and P is the lowest data value. Like accuracy, precision is always a positive number, so we use an absolute value here as well.
A product test can be accurate but imprecise, or precise but inaccurate. The goal is always the combination of accuracy and precision. In order for engineers to make a recommendation based on experimental test data, it is important to determine how a device measures against a standard. Engineers need these benchmarks to determine the quality of a product.
By determining the percent accuracy of an experiment, and comparing this percentage to a standard, you can determine, based on experimental evidence, whether the prototype has passed or failed. The equation for percent accuracy is:
In this equation, %Acc is the percent accuracy, PS is the standard value, and AP is the value you measured.
Percent accuracy allows an engineer to determine how precise a device is compared to the standard. The equation for percent precision is:
%Prec = 100e-BP
In this equation, %Prec is the percent precision, B is the exponential decay factor, and P precision you measured.
Calculating percent accuracy and percent precision and then comparing your results to a standard will allow you to determine the quality of your prototype.
3 YOUR ASSIGNMENT
Individual Lab Report
Follow the lab report guidelines laid out in the page called Specifications for Writing Your Lab Reports in the Technical Communication section of this manual. As you write, the following discussion points should be addressed in the appropriate section of your lab report:
- Discuss product evaluation and the importance of testing.
- Explain the importance of designing to a standard.
- Discuss average, accuracy, and precision.
- Assess the techniques used in the testing process.
- Discuss better testing methods for the robot design.
- Make a final evaluation of the product. Describe its accuracy and precision.
- Discuss the importance of % precision and % accuracy in determining product quality.
- State your recommendations.
Team PowerPoint Presentation
Follow the presentation guidelines laid out in the page called EG1004 Lab Presentation Format in the Introduction to Technical Presentations section of this manual. When you are preparing your presentation, consider the following points:
- Based on the test results, what is your determination for the product's marketability?
- If determinations were negative, what improvements could be made?
- Why is product evaluation important to you?
4 MATERIALS AND EQUIPMENT
- Minstorms Kit
- Computer with Mindstorms Software
- Protractor and Ruler
Remember: You are required to take notes. Experimental details are easily forgotten unless written down. EG Standard Note Paper can be downloaded and printed from the EG website the EG1004 Web site. Use your lab notes to write the Procedure section of your lab report. At the end of each lab your TA will scan your lab notes and upload them to the EG1004 course section on MyPoly. You must attach your lab notes at the end of your lab report (use the "Insert Object" command in MS Word after your Conclusion). Keeping careful notes is an essential component of all scientific practice.
5 PROCEDURE
Problem Statement
You have been hired by an engineering firm to build and evaluate a robotics kit by building a specified robot design. You have been asked to test the robot design and the overall kit, and to make suggestions for design improvements.
- Build a robot according to the lab handout provided by your TA.
- Create a program in Mindstorms:
- Start the Mindstorms Program. On the welcome screen type in a name for your program and press Go.
- Upload a program that will set the robot to travel for 4 seconds
- Measure the distance traveled.
- Repeat this test 10 times.
- Use a protractor to draw a 0° line from the start point.
- Mark the start point and place your robot there. Identify a reference point on the robot.
- Run the Distance Test.
- Make a mark at the point where the robot finished, and draw a line from the start point to this mark.
- Measure the positive angle of deviation.
- Repeat this procedure 10 times.
- Write the program in Mindstorms to perform this test. Your TA will explain this process.
- Upload your rotation program from the computer to the NXT.
- Make a center point on a sheet of paper.
- Run the program for 5 seconds.
- Mark the reference point after rotation and measure the rotation angle.
- Divide this value by 5 to determine the angle of rotation for 1 second.
- Multiply the angle of rotation per second by 4 to obtain the standard.
- Go back to the Mindstorms program and change the time from 5 seconds to 4 seconds.
- Upload the new program to the NXT.
- Run the program 10 times. Make sure to measure the angle the robot rotated by each time.
- If PS was determined to be 5cm and AP was found to be 7cm, then the percent accuracy (%Acc) is 60%. This test does not pass the 80% criterion.
- If PS was determined to be 5 feet and AP was found to be 25 feet, then the result of the equation is -300%. Since a negative percentage has no meaning in engineering, it can be said that the product has a percent accuracy of zero. Any percent accuracy less than zero is considered to be equal to zero.
- If PS was determined to be 10 ft, and AP was found to be 8 ft, then the result of the equation is exactly 80%, which is a passing result.
- For the Distance Test, if the precision (P) was measured to be 0.5 cm, the percent precision (%Prec) is 95.3%. This robot passes the 80% criterion for this test.
- For the Angle of Deviation Test, if the precision (P) was measured to be 10.25°, the percent precision (%Prec) is 71.8%. This robot does not pass the 80% criterion.
b. Click on Complete Pallet to see all the programming icons.
Turn off Robot Educator by clicking the small X.
<img src="lab3_files/image005.jpg">
c. Your screen should look like this. You are now ready to write a program.
<img src="lab3_files/image006.jpg">
d. Click on Actions, then on the Motor icon to select it.
<img src="lab3_files/image007.jpg">
e. Place the motor icon on to the “start�? square.
<img src="lab3_files/image008.jpg">
f. Click the motor icon to see its properties. Here you can define exactly how you want the motor to spin and when to stop.
<img src="lab3_files/image009.jpg">
g. Continue writing your program by placing icons after each other. The execution of your program will run from left to right.
In the sample program below, motor A will spin forward for 5 seconds, Stop, Wait 5 seconds, and then spin backwards for another 5 seconds.
<img src="lab3_files/image010.jpg">
h. For this lab you can use the Move icon to easily instruct your robot to move.
<img src="lab3_files/image011.jpg">
i. Whenever you need to, you can use Robot Educator to quickly learn how to program in Lego Mindstorms.
<img src="lab3_files/image012.jpg">
Note: Program and test your robot for each test specified. To do this, make a list of the tests in your lab notebook. Brainstorm what you want the robot to do in each test. Then, use your notes to write your program. Record the standard, average, accuracy, and precision. If your robot does not operate correctly, troubleshoot the problem. Your TA must initial your original data.
Distance Test
Note: Use the Metric system for this test
In this test, the robot will go forward a set distance in a specified time. You are to test the robot's accuracy and precision for this distance. Before testing begins the standard must be found. To find the standard, program the robot to go forward for 5 seconds and measure the distance it travels. Divide your result by 5, and then multiply by 4. This number is your standard. Once the standard has been obtained, the testing can begin.
Warning: Read the section on the Angle of Deviation Test before performing this test.
Angle of Deviation Test
Note: This test can be performed at the same time as the Distance Test. Since the robot is expected to travel in a straight line, the standard is 360°.
In this test, the robot will move forward for 4 seconds and the angle of deviation will be recorded. The angle of deviation is the angle between a straight line and a line drawn from the starting point to where the robot finishes.
Rotation Test
In this test, the robot will rotate around an axis. Again, the standard must be obtained. By running the rotation test for 5 seconds and dividing the angle obtained by 5, you will determine how many degrees your robot rotates per second. This is the standard. With the standard in mind, project how many degrees your robot will rotate in 4 seconds.
You now have measurements for each test you have performed. You must analyze this data. Calculate the average, accuracy, and precision for each test. Compare your results to the standard in order to determine if your robot is accurate and precise. You must also calculate percent accuracy and percent precision. Use this formula to calculate percent accuracy:
<a href="https://egmanual.poly.edu/index.php?title=Image:Lab_hardsyn_13.gif" title="image:lab_hardsyn_13.gif"><img src="lab3_files/image003.gif" ></a>
In this equation, %Acc is the percent accuracy, PS is the standard value, and AP is the value you measured.
Examples:
Use this formula to calculate % precision:
%Prec = 100e-BP
In this equation, %Prec is the percent precision, B is the exponential decay factor, and P precision you measured. Each individual test requires the use of a different exponential decay factor (B):
Distance Test: B = 0.0972
Angle of Deviation Test: B = 0.0323
Rotation Test: B = 0.0093
Examples:
Tabulation of Results
To decide if the product is ready to manufacture, arrange your data in a table designed to calculate its precision and accuracy. Remember it must pass all the tests at a rate of at least 80%.
Accuracy
%Acc
Pass/Fail
Precision
%Prec
Pass/Fail
Distance Test
Angle of Deviation Test
Rotational Test
Your lab work is now complete. Please clean up your workstation. Return all unused materials to your TA. Refer to section 3 Your Assignment for the instructions you need to prepare your lab report.
Footnotes
1 Oakes, W.C., L.L. Leone, and C.G. Gunn, Engineering Your Future w:st="on">MI: Great Lakes Press, 2002
</html>
Warning: Do not disassemble the robot when you finish. You will use it again in the next lab.