DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Notice to Applicants
2. This communication is in response to the application filled on 02/01/2024.
3. Claims 1-10 are pending.
4. Limitations appearing inside {} are intended to indicate the limitations not taught by said prior art(s)/combinations.
Information Disclosure Statement
5. The information disclosure statements (IDS) submitted on 02/01/2024, 12/20/2024 and 02/04/2025 have been considered by the examiner.
Election/Restrictions
6. Applicant's election with traverse of Group I in the reply filed on 03/09/2026 is acknowledged. The traversal is on the grounds that Group I claims recite a method and claim 3 recites a product used in the method of claim 2. This is not found persuasive because Group I and Group II lack unity of invention. Group II specifically lacks the technical feature of determination of a terminal grade based on reference grade data as recited in Group I (see MPEP 2111.02 with regards to preamble statements reciting purpose or intended use), and Group I lacks the technical features of the structural relationship between images, the conveyor belt and its movement direction, mirrors/gaps in a conveyor belt, and the reversing mechanism. Furthermore, Group I fails to recite simultaneous opposite side imaging and a second conveyor. Additionally, applicant's arguments filed 03/09/2026 have been fully considered but they are not persuasive. Applicant’s arguments rely on language solely recited in preamble recitations in claims of Group II. When reading the preamble in the context of the entire claim, the recitation of the claims as to be used with the method of Group I is not limiting because the body of the claim describes a complete invention and the language recited solely in the preamble does not provide any distinct definition of any of the claimed invention’s limitations. Thus, the preamble of the claims is not considered a limitation and is of no significance to claim construction. See Pitney Bowes, Inc. v. Hewlett-Packard Co., 182 F.3d 1298, 1305, 51 USPQ2d 1161, 1165 (Fed. Cir. 1999). See MPEP § 2111.02.
The requirement is still deemed proper and is therefore made FINAL.
Claim Rejections - 35 USC § 103
7. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
8. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
9. Claims 1 is rejected under 35 U.S.C. 103 as being unpatentable over U.S. Publication No. 2021/0116392 to Fitzgerald et al. (hereinafter Fitzgerald) in view of 2020/0265487 to Silva et al. (hereinafter Silva).
10. Regarding Claim 1, Fitzgerald discloses a method for grading a used terminal, comprising ([par. 0002, ln. 1-8] “…systems and methods for detecting fault conditions in mobile devices. More particularly, the present invention provides for systems and methods for detecting a that a mobile device has a fault such as cracked, scratched, chipped, or otherwise damaged screen or housing, and reporting the status of the screen or housing, working or not, so that appropriate action may be taken by a third party”):
moving a terminal in a plane ([par. 0133, ln. 1-38] “Gravity assist may also be used to move the mobile device under test to and/or through the test fixture; further, gravity may be used through a standard conveyer belt to flip the device over to obtain an image of the back of the mobile device… the device under test could be in motion, and could be transferred down a conveyer belt and take its own image as it moves along the belt. The continued or continual motion could lead to significant gains in processing time in a warehouse/recycler environment… the conveyer could pause long enough to take a picture… a lever could be dropped in to the conveyer belt to flip the mobile device. Further, a multi-camera setup such as the illustrations of FIGS. 14, 15, and 15A may be used to capture multiple images and angles of the mobile device (and potentially build a 3-D model of the exterior view of the mobile device. The mobile device may proceed down the conveyor to come in the box shown in FIGS. 15 and 15A, the mobile device passes through a screen such as a black curtain, the curtain comes down, lights come on, camera gets photographed, captured images are then uploaded, and all tests conducted… a conveyer could be used to capture a front facing picture in a reflecting surface, the conveyer drops/flips the mobile device, then the back picture is taken the back facing camera…Further, in a conveyer environment (or in the 3-D photo box of FIGS. 15-15A, external cameras may capture side images of the mobile device. External lighting effects may be applied as well as discussed above, and as shown in the lighting examples of FIGS. 9-13.”);
irradiating a front surface or a back surface of the moving terminal with illumination light from diagonally above {a rear of the terminal toward a front}, and capturing an image of an illuminated part of the terminal with a camera directed from diagonally above a front of the terminal toward the rear ([par. 0133, ln. 1-38], [Fig 9, LED Strip and RasberryPi Camera], [Fig. 10-13] see diagonal capturing of image of illuminated part of the terminal, [Fig. 15, see backside cameras, camera on top, see also LED lighting], [par. 0121, ln. 1-14] “FIGS. 9-13 illustrate… a mobile device 800 is externally illuminated by a light source, such as an LED light strip… the colors of the lights may be selectively changed to illuminate the mobile device and enhance crack detection and analysis… the lights need to be in a position where they will cause scatter from the crack if present in the glass. An LED strip with RGB lighting may be utilized in various embodiments so that color and intensity of the light can be changed as required as multiple lights may be needed depending on the number of cameras used.”);
referring image data of the terminal to a plurality of types of reference grade data ([par. 0140, ln. 1-36] “…using a neural network analysis approach, systems and a methods are provided to identify a crack or other kind of defect associated with a mobile device through image analysis… the user of the mobile device 800 initiates the device defect test 1615… FIG. 17, an image of the display of the mobile device 800 is captured 1620, and the captured image is optionally pre-processed 1625 by rotating the image to a portrait mode and/or fine rotating to reduce skew, and/or cropping the image to a desired aspect ratio. The captured (and optionally pre-processed) image is then uploaded 1630 to a server (such as server 860), and as discussed further in regards to FIG. 18, the uploaded image is processed 1635 to determine whether any cracks or other types defects are present… the extent and characteristics of the screen cracks or defects are characterized by the processing step 1635 for further analysis and reporting. Such analysis of the captured image (and possibly processed) image from the mobile may include determining whether the captured image comprises one or more defect indicators. Such defect indicators may be pre-defined within a pre-trained neural network into defect classes such as cracks, chips, fractures, mars, wear, scratches, abrasions, engravings, stuck pixels, dead pixels, color errors, color bleed, errant markings and the like. Based on this analysis, a cosmetic defect grading of the mobile device may be produced, which may be indicative of the relative overall visual state of the mobile device. Such grading may be useful in calculating a resale value of the mobile device and/or determining an insurance value or insurability criterion related to the mobile device.”), each sorted {by grade} ([par. 0150, ln. 1-31] “Step 1830 illustrates the analysis of the rescaled sub-image by a neural network to identify defects … The rescaled and perspective-transformed sub-image (for instance, 1400A in FIG. 20A) is applied to the input of a convolutional neural network which has been previously trained to detect defects with training data comprising displays and accompanying defect classes. Once the network analyzes the image, an output is produced that identifies a defect, such with a colored polygon 1900A shown in FIG. 20B. The colored polygon draws attention to the defect found by the neural network, and as mentioned previously, additional outputs and analysis such as heat maps may be provided based on the particular network architecture. More illustrative examples follow in FIGS. 20C-E, where input sub-image (FIG. 20C, 1400A) with defect (FIG. 20C, 1900) is input to neural network 47 of the present invention, to produce an output (FIG. 20E, 1400B) with defect circumscribed by a polygon (FIG. 20E, 1900A), and a heat map output version (FIG. 20D, 1400D), depicting a heat map 1900D corresponding to activation classes for detected defects (FIG. 20C, 1900) in the display sub-image (FIG. 20C, 1400A)...”), stored in a database of a server ([par. 0105, ln. 18-21] “The database 880 may also store neural network training data, wherein exemplary images of defects in mobile devices are correspondingly associated with identified defect classes, outputs or states.”), the image data of the terminal being formed by processing the image ([par. 0140, ln. 1-36], [par. 0150, ln. 1-31]);
determining a grade (rank) of reference grade data, to which the image data is determined to belong, to be a grade of the image data ([par. 0140, ln 1-36], [par. 0150, ln. 1-31]); and
outputting the determination result ([par. 0140, ln 1-36], [par. 0141, ln. 1-6] “Once the analysis 1635 of any potential defects is complete, the resulting information is formatted 1640 for presentation, storage, or display to any user for any necessary purpose, such as for assessment of whether the mobile device 800 is in proper working condition. The process then exits 1645.”).
Fitzgerald does not specifically disclose wherein the illumination light is above a rear of the terminal toward a front, though one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that such an arrangement would have been obvious in view of the disclosure of multiple cameras and various arrangements of the cameras, which would require different arrangements for lighting ([par. 0121, ln. 1-14], [par. 0133, ln. 1-38]). Furthermore, Fitzgerald does not specifically disclose wherein the reference grade data is sorted by grade, though one of ordinary skill in the art would recognize that Fitzgerald does disclose wherein it is sorted by defect class ([par. 0150, ln. 1-31]).
However, Silva discloses illumination light sources above a rear of a terminal toward a front ([Fig. 9A, 901 a and 901b, Fig. 9B, 941a and b and 931a and b], [par. 0035, ln. 1-31] “The one or more light sources includes two sets of light sources that are arranged orthogonal to each other. Because the cracks and/or scratches on the mobile device 150 can run in different directions (e.g., both horizontally and/or vertically), having two sets of orthogonally arranged light sources allows the cameras to capture various combinations of the cracks and/or scratches. For example, a first angle between light beams from one set of lights and the top side of the inspection plate 144 can be between 30 to 60 degrees (e.g., preferably 45 degrees) while a second angle between light beams from a second set of lights and the left side of the inspection plate 144 can be between 30 to 60 degrees (e.g., preferably 45 degrees). The two sets of lights are positioned orthogonal to each other. FIG. 9B illustrates an example arrangement of two sets of light sources in accordance with some embodiments of the present technology. A first set of light sources 931a,b is arranged orthogonally with a second set of light sources 941a,b. Light beams 951a,b from the first set of light sources 931a,b are about 45 degrees from either side of the inspection plate 144 (e.g., X axis and/or Y axis). Similarly, light beams 961a,b from the second set of light sources 941a,b are about 45 degrees from either side of the inspection plate 144 (e.g., X axis or/or y axis). Such arrangement can help reduce or eliminate imaging noise or shadows from other components of the kiosk 100 that are arranged along the sides of the inspection plate… additional sets of light sources can be arranged within the upper and/or lower chamber to reveal damage that can not be visible from orthogonal arrangements of the light sources.”), and wherein the reference grade data is sorted by grade ([par. 0062, ln. 1-16] “FIG. 11 illustrates an example architecture 1100 of training a neural network in accordance with some embodiments of the present technology. As shown in FIG. 11, the neural networks can be trained using pre-collected images 1101 which have been labeled by inspectors 1103 (e.g., human inspectors, electronic labeling systems, etc.). In some embodiments, images in the training set are each associated with a cosmetic evaluation indication (e.g., “cosmetically good” or “cosmetically bad”) agreed on by at least a threshold number of inspectors (e.g., two human inspectors). Therefore, the training set includes representative images of electronic devices in a particular cosmetic status that a threshold number of inspectors have agreed are, and the cosmetic status can be reasonably determined by visual inspection without requiring presence of the device phone on site.”, [par. 0063, ln. 1-18] “The training set can include images that have been pre-processed the same way as would image(s) that contribute to the input of the machine learning system 1105 (e.g., neural network(s)) once it is deployed. The training set can include equal-sized or substantially equal-sized (e.g., within 5%, 10%, or 15% difference in size) subsets of images associated with each distinct cosmetic evaluation indication. For example, for approximately 700,000 images used in training, about 350,000 are associated with a “cosmetically good” indication and the other 350,000 are associated with a “cosmetically bad” indication. Dividing the training set in this manner can prevent or mitigate certain “random guess” effects of trained neural network(s), where an output can be biased to favor those reflected by a larger portion of the training set.”). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed inventio, would recognize Fitzgerald and Silva as within the same filed of grading/defect detection for used terminals using neural networks, and as analogous to the claimed invention. The motivation to combine the light source placement and of Silva is disclosed in Silva, wherein placing the light source above a rear of a terminal toward a front allows reduction in shadows and image noise and inspection of defects not visible in other light source arrangements ([Fig. 9A, 901 a and 901b, Fig. 9B, 941a and b and 931a and b], [par. 0035, ln. 1-31] “…Such arrangement can help reduce or eliminate imaging noise or shadows from other components of the kiosk 100 that are arranged along the sides of the inspection plate… additional sets of light sources can be arranged within the upper and/or lower chamber to reveal damage that can not be visible from orthogonal arrangements of the light sources.”). The motivation to combine the reference grade data sorting by grade of Silva is likewise disclosed in Silva, wherein it prevents/mitigates faulty training of the neural networks ([par. 0063, ln. 1-18] “…Dividing the training set in this manner can prevent or mitigate certain “random guess” effects of trained neural network(s), where an output can be biased to favor those reflected by a larger portion of the training set.”). One of ordinary skill in the art, before the effective filling date of the claimed invention, would have combined method of Fitzgerald with the light source placement and reference data sorted by grade of Silva, through known means, with no change to their respective function, and the combination would have yielded nothing more than predicable results.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Fitzgerald with the light source placement and reference data sorted by grade of Silva to obtain the invention as specified in claim 1.
11. Regarding Claim 2, a combination of Fitzgerald and Silva teaches the method of claim 1. Fitzgerald further discloses wherein image data of both a front and back surface of the termina is acquired by: irradiating the terminal being moved with the illumination light and capturing an image of the terminal with the camera when the terminal is moved on a belt conveyor with the front surface or the back surface facing upward ([par. 0133, ln. 1-38]); the reversing the terminal and moving the terminal with the back surface or front surface facing upward ([par. 0133, ln. 1-38]); and irradiating the terminal with another illumination light and capturing an image of the terminal with another camera ([par. 0133, ln. 1-38]). Specifically, one of ordinary skill in the art, before the effective filling date of the claimed invention, would recognize that “flip” using a lever and/or gravity as disclosed in Fitzgerald to be analogous to a reversing of the terminal. Specifically, the examiner notes that as described in Fitzgerald, after flipping, the front or back surface would be facing upward ([Fig. 15 and 15A]). With regard to “another illumination light” and “another camera” the examiner highlights it would have been obvious to include another camera, since Fitzgerald discloses multiple cameras and multiple illumination lights ([Fig. 15], [par. 0121, ln. 1-14], [par. 0133, ln. 1-38]). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filling date of the claimed invention, to combine the method of Fitzgerald with the light source placement and reference data sorted by grade of Silva to obtain the invention as specified in claim 2.
Conclusion
12. The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. See PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULO ANDRES GARCIA whose telephone number is (703)756-5493. The examiner can normally be reached Mon-Fri, 8-4:30PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on (571)272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULO ANDRES GARCIA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669