DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/29/2026 has been entered.
Response to Arguments
Applicant's arguments, see Remarks pages 1-5, filed 05/29/2026, with respect to the rejections of claim(s) 1-8 under 35 U.S.C. 101 have been fully considered and are persuasive. The rejections of claim(s) 1-8 have been withdrawn.
Applicant’s arguments, see Remarks page 6, filed 05/29/2026, with respect to the rejection of claim(s) 1-8 under 35 U.S.C. 103, specifically in regards to the claim 1 limitation “storing a default ranked set of ROIs based on the respective recurrence frequencies”, have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below).
Applicant's remaining arguments, see Remarks pages 5-10, filed 05/29/2026, with respect to the rejections of claim(s) 1-8 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
On pages 6-7 of Remarks, Applicant argues:
PNG
media_image1.png
644
795
media_image1.png
Greyscale
Examiner respectfully disagrees.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Wherein the limitation “wherein each of the first and the second ROIs are less than the entire FOV” is taught by prior art reference Al Rashdan et al. (US-20220383541-A1), as further disclosed in the rejection of claim 1 under 35 U.S.C. 103 below.
Paragraph 0079 of Petrou discloses “The system and method may further use the location of objects in different frames to determine whether the objects are the same or different items. FIGURE 14 illustrates a sequence of frames 1411 and 1421. The processor detects three shapes in the first image, namely, the bottleshape, logo and bar code. The processor further determines a bounding box 1412-14 for each shape. In the next frame 1421, the processor detects additional objects and determines a bounding box 1422-24 for each. The processor may determine that the regions defined by three of the bounding boxes, namely bounding boxes 1422-24, contain objects that are visually similar to the prior frame. FIGURE 15 superimposes the bounding boxes of the three pairs of visually similar objects of frames 1411 and 1421 relative to the edges of the frames. All of the regions defined by the bounding box overlap. Accordingly, the processor may determine that the objects are likely associated with the same item, i.e., the same bottle instead of two different bottles.”. Wherein, in order to determine whether an object is present within an image, the bounding boxes of the objects, across the frames, are analyzed based on object similarity and whether the bounding boxes overlap. If determined to overlap and meet the similarity requirements, the object is determined to be recurring across the analyzed frames, and is added to the weight calculation disclosed in 0070 of Petrou.
In addition, the weight calculation disclosed in 0070 of Petrou, which recites “The system and method may also weigh information obtained from the most recent frames more heavily than information obtained from older frames. For instance, when preparing a query based on the frequency of descriptions across three of the most recent frames, the processor may give an object a relative weight of 1.00 if the object only appears in the most recent frame, a weight of 0.25 if the object only appears in the oldest frame, and a weight of 1.75 (equal to 1.00+0.50+0.25) if the object appears in all three frames. The system and method may determine and weigh other signals than those described herein.”, constitutes the generation of a recurrence frequency for each of the ROIs and the storing of a default ranked set of ROIs based on the respective recurrence frequencies, since each object is weighed based on the object’s frequency within the frames analyzed and each of the object’s weights is compared to each other in order to determine the highest weighted object.
Thus, Petrou discloses stable, pre-defined regions of the FOV across which recurrence statistics are accumulated to produce a stored default ranking.
On page 7 of Remarks, Applicant argues:
PNG
media_image2.png
519
793
media_image2.png
Greyscale
Applicant’s arguments with respect to the claim 1 limitation “ranking the first and second ROIs includes storing a default ranked set of ROIs based on the respective recurrence frequencies” have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below).
In addition, in response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Wherein the limitations “wherein ranking the first and second ROIs includes applying each of the first and second ROIs to multiple images, analyzing the multiple images to determine whether the visual feature is located within each of the first and the second ROIs, generating a recurrence frequency for each of the first and second ROIs, and storing a default ranked set of ROIs based on the respective recurrence frequencies.” are taught by prior art reference Petrou, as further disclosed in arguments above and in the rejection of claim 1 under 35 U.S.C. 103 below.
On pages 7-8 of Remarks, Applicant argues:
PNG
media_image3.png
334
794
media_image3.png
Greyscale
Applicant’s arguments with respect to the claim 1 limitation “wherein each of the first and the second ROIs are less than the entire FOV” have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below).
On pages 8-9 of Remarks, Applicant argues:
PNG
media_image4.png
984
810
media_image4.png
Greyscale
Applicant’s arguments with respect to the claim 1 limitations: “ranking the first and second ROIs includes storing a default ranked set of ROIs based on the respective recurrence frequencies” and “wherein each of the first and the second ROIs are less than the entire FOV”, have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below).
On pages 9-10 of Remarks, Applicant argues:
PNG
media_image5.png
423
817
media_image5.png
Greyscale
In addition, Applicant’s arguments with respect to the combination of prior art references Petrou, Ray, and Scott reflecting improper hindsight reasoning have been fully considered and are moot in view of the new grounds of rejection (detailed in the rejections below).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Petrou et al. (WO-2013085985-A1) hereinafter referenced as Petrou, in view of Ray et al. (US-20060261167-A1) hereinafter referenced as Ray, and Al Rashdan et al. (US-20220383541-A1) hereinafter referenced as Al Rashdan.
Regarding claim 1, Petrou discloses: A method for operating a machine vision system (Petrou: Abstract), the machine vision system including a computing device for executing an application and a fixed imaging device communicatively coupled to the computing device (Petrou: Figure 1; 0038: “As shown in FIGURE 3, camera 163 may be disposed on the back side of the device. The camera angle may be fixed relative to the orientation of the device.”), the method comprising:
(a) capturing, via the fixed imaging device, a first image over a field of view (FOV) (Petrou: Figure 9: Frame 911; 0065: “FIGURE 9 illustrates three frames 911, 921 and 931 taken in sequence.”);
(b) analyzing, via the application, at least a portion of the first image to detect a visual feature within the first image (Petrou: Figure 9: Features 913-916; 0065: “the processor has detected and recognized a number of objects in frames 911, 921 and 931. Specifically, the processor detected features 913, 923 and 933…the processor recognized features 914, 924 and 934 as corresponding with text, features 915 and 925 as corresponding with bar codes, and features 916, 926 and 936 as corresponding with a logo.”);
(c) determining, via the application, a location of the visual feature within the first image; (d) determining, via the application, a first region of interest (ROI) within the first image based on the location of the visual feature (Petrou: Figure 12: Bounding boxes 1215-1217; 0065: “the processor has detected and recognized a number of objects in frames 911, 921 and 931. Specifically, the processor detected features 913, 923 and 933…the processor recognized features 914, 924 and 934 as corresponding with text, features 915 and 925 as corresponding with bar codes, and features 916, 926 and 936 as corresponding with a logo.”; Wherein detected features are surrounded in bounding boxes);
(e) capturing, via the fixed imaging device, a second image (Petrou: Figure 9: Frame 921; 0065: “FIGURE 9 illustrates three frames 911, 921 and 931 taken in sequence.”);
(f) analyzing, via the application, at least a portion of the second image to detect the visual feature within the second image (Petrou: Figure 9: Features 923-926; 0065: “the processor has detected and recognized a number of objects in frames 911, 921 and 931. Specifically, the processor detected features 913, 923 and 933…the processor recognized features 914, 924 and 934 as corresponding with text, features 915 and 925 as corresponding with bar codes, and features 916, 926 and 936 as corresponding with a logo.”);
(g) determining, via the application, a location of the visual feature within the second image; (h) determining, via the application, a second ROI within the second image based on the location of the visual feature (Petrou: Figure 12: Frame 1212; 0065: “the processor has detected and recognized a number of objects in frames 911, 921 and 931. Specifically, the processor detected features 913, 923 and 933…the processor recognized features 914, 924 and 934 as corresponding with text, features 915 and 925 as corresponding with bar codes, and features 916, 926 and 936 as corresponding with a logo.”; Wherein detected features, once detected, are surrounded in bounding boxes);
(i) ranking the first and second ROIs (Petrou: Figure 10; 0070: “The system and method may also weigh information obtained from the most recent frames more heavily than information obtained from older frames. For instance, when preparing a query based on the frequency of descriptions across three of the most recent frames, the processor may give an object a relative weight of 1.00 if the object only appears in the most recent frame, a weight of 0.25 if the object only appears in the oldest frame, and a weight of 1.75 (equal to 1.00+0.50+0.25) if the object appears in all three frames. The system and method may determine and weigh other signals than those described herein.”;
0079: “The system and method may further use the location of objects in different frames to determine whether the objects are the same or different items. FIGURE 14 illustrates a sequence of frames 1411 and 1421. The processor detects three shapes in the first image, namely, the bottleshape, logo and bar code. The processor further determines a bounding box 1412-14 for each shape. In the next frame 1421, the processor detects additional objects and determines a bounding box 1422-24 for each. The processor may determine that the regions defined by three of the bounding boxes, namely bounding boxes 1422-24, contain objects that are visually similar to the prior frame. FIGURE 15 superimposes the bounding boxes of the three pairs of visually similar objects of frames 1411 and 1421 relative to the edges of the frames. All of the regions defined by the bounding box overlap. Accordingly, the processor may determine that the objects are likely associated with the same item, i.e., the same bottle instead of two different bottles.”;
Wherein the ranking of the features contained in the ROIs across the frames constitutes ranking the ROIs of the frames);
(j) capturing, via the fixed imaging device, a third image (Petrou: Figure 9: Frame 931; Figure 16; 0082: “if a bottle of Brand OR Bleach appears in ten frames in a row, it may be more efficient to make a single query for the product and track its presence in the frames instead of making ten different queries and ranking an aggregated list of ten different results.”; Wherein images are iteratively captured and processed.);
(k) analyzing, via the application, a third ROI within the third image to detect the visual feature (Petrou: 0082: “By tracking those objects that are associated with the same item from frame to frame, or within a single frame, the system and method can avoid duplicative searches and apply greater or lesser weights to the information used during a search. For instance, as noted above, the fact that the same item appears in multiple frames may be an indication that the item is of interest to the user.”); and
(l) responsive to detecting the visual feature in the third image, transmitting data associated with the visual feature in the third image to a host processor (Petrou: 0072: “A processor may select a subset of the returned results and display the selected subset to the user. This may include selecting the highest ranking result as the optimum annotation…The processor may also select as the optimum annotation the information that appears most applicable to the type of the recognized object, i.e., the address of a building if a building is recognized or a person's name if a person is recognized”; Wherein the higher/highest ranked results are processed and selected by the processor and transmitted to the user);
wherein ranking the first and second ROIs includes applying each of the first and second ROIs to multiple images, analyzing the multiple images to determine whether the visual feature is located within each of the first and the second ROIs (Petrou: 0079: “The system and method may further use the location of objects in different frames to determine whether the objects are the same or different items…The processor may determine that the regions defined by three of the bounding boxes, namely bounding boxes 1422-24, contain objects that are visually similar to the prior frame. FIGURE 15 superimposes the bounding boxes of the three pairs of visually similar objects of frames 1411 and 1421 relative to the edges of the frames. All of the regions defined by the bounding box overlap. Accordingly, the processor may determine that the objects are likely associated with the same item, i.e., the same bottle instead of two different bottles.”), generating a recurrence frequency for each of the first and second ROIs, and storing a default ranked set of ROIs based on the respective recurrence frequencies (Petrou: Figure 10; 0070: “The system and method may also weigh information obtained from the most recent frames more heavily than information obtained from older frames. For instance, when preparing a query based on the frequency of descriptions across three of the most recent frames, the processor may give an object a relative weight of 1.00 if the object only appears in the most recent frame, a weight of 0.25 if the object only appears in the oldest frame, and a weight of 1.75 (equal to 1.00+0.50+0.25) if the object appears in all three frames. The system and method may determine and weigh other signals than those described herein.”;
Wherein the weighting of the visually similar objects at the same location including the object frequency within the analyzed frames constitutes the generation of a recurrence frequency for each of the ROIs.).
Petrou does not disclose expressly: analyzing, via the application, a third ROI within the third image to detect the visual feature, the third ROI within the third image being based on the ranking.
Ray discloses: an automatic data collection device for the identification of a target symbol, wherein the device is able to decode the identified target and rank, sort, or prioritize the detected targets (Ray: Abstract). Wherein the device further comprises the determination of an order for a set of potential targets based on weighed criteria, such as such as the target’s position, target’s size, and history of the collected/processed symbols (Ray: 0013: “The methods and systems emphasize or highlight a number of the potential targets in a ranked, sorted or prioritized order. The number may be user configurable. The order may be based on one or more criteria, for example: particular symbology, type of symbology (e.g., one-dimensional versus two-dimensional), enabled symbologies (i.e., symbologies which the reader is currently programmed to decode), symbologies of the most recently decoded symbols, position of the potential target in the field-of-view, size of the potential target, or may be based on other criteria. Some of the criteria may be user configurable, and some criteria may be discerned by the methods and systems based on prior use of the reader and/or based on prior use by the user. The criteria may be weighted.”;
0112: “the ranking, sorting, ordering or prioritization may be based on a preference for representations appearing in a certain portion of the field-of-view as opposed to other portions. Thus, for example, regions of interest appearing in the center of the field-of-view 14 may be given preference over regions appearing in other portions of the field of view 14…certain regions of the field-of-view 14 may be defined as having low or no priority. Thus, for example, the user may define a region that should be ignored or the machine-readable symbol reader 12 may determine that a particular region such as the lower left corner of the field-of-view 14 should consistently be ignored based on the history of prior symbol acquisition and/or decoding.”). Upon determining the ranking order of the potential targets, a new image’s potential target is selected to be the potential target with the highest ranking (Ray: 0019: “In yet a further aspect, the methods and apparatuses may process the most highly ranked potential target if no user input is received within a set period of time.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the selection and ranking of potential targets based on weighed criteria taught by Ray to select ROIs in a current frame disclosed by Petrou based on weights and locations of processed features within previous images. The suggestion/motivation for doing so would have been “The methods and systems emphasize or highlight a number of the potential targets in a ranked, sorted or prioritized order. The number may be user configurable. The order may be based on one or more criteria, for example: particular symbology, type of symbology (e.g., one-dimensional versus two-dimensional), enabled symbologies (i.e., symbologies which the reader is currently programmed to decode), symbologies of the most recently decoded symbols, position of the potential target in the field-of-view, size of the potential target, or may be based on other criteria. Some of the criteria may be user configurable, and some criteria may be discerned by the methods and systems based on prior use of the reader and/or based on prior use by the user. The criteria may be weighted..” (Ray: 0013; Wherein the potential targets are ranked and selected based criteria to determine how useful targets are.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Petrou in view of Ray does not disclose expressly: wherein each of the first and the second ROIs are less than the entire FOV.
Al Rashdan discloses: a method for detecting visual features within a captured image (Al Rashdan: 0058: “Upon detecting a visual feature 106 in the field of view (FOV) of a camera of vehicle 104, visual feature 106 is identified in an image, a bounding box is positioned around visual feature 106 in the image (e.g., see FIG. 5 including visual feature 500 and bounding box 502 ), visual feature 106 is extracted out of the image, visual feature 106 is decoded for its data”;
0068: “FIG. 7 is a flowchart of an example method 700 of detecting codes in an image, in accordance with various embodiments of the present disclosure. ”). Wherein the determined region of interest for each visual feature is less than the imaging device’s field of view (Al Rashdan: 0070: “a model (e.g., of DL module 608 of FIG. 6 ) may be run on the converted image object (i.e., to detect one or more codes (e.g., QR codes)), and method 700 may proceed to block 706 . At block 706, it may be determined whether one or more codes are detected. If the model was able to detect a code, a bounding box may be generated (e.g., at least partially around the detected code), and method 700 may proceed from block 706 to block 708, where a bounding box region is expanded (e.g., to improve detectability). For example, a bounding box region may be expanded (e.g., by a percentage p, such as between 5% and 20%) to ensure edges of the code do not fall outside an original bounding box region.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the bounding box expansion taught by Al Rashdan to expand the determined bounding boxes disclosed by Petrou in view of Ray. The suggestion/motivation for doing so would have been “a bounding box region may be expanded (e.g., by a percentage p, such as between 5% and 20%) to ensure edges of the code do not fall outside an original bounding box region” (Al Rashdan: 0070). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Petrou in view of Ray with Al Rashdan to obtain the invention as specified in claim 1.
Regarding claim 2, Petrou in view of Ray and Al Rashdan discloses: The method of claim 1, wherein the analyzing the third ROI includes: setting the third ROI to be a higher ranked ROI from the first ROI and the second ROI (Petrou: 0082: “By tracking those objects that are associated with the same item from frame to frame, or within a single frame, the system and method can avoid duplicative searches and apply greater or lesser weights to the information used during a search. For instance, as noted above, the fact that the same item appears in multiple frames may be an indication that the item is of interest to the user.”; Wherein the ROIs of the features in the third image are initially tracked from their previous position in the previous frame)
(Ray: 0112: “the ranking, sorting, ordering or prioritization may be based on a preference for representations appearing in a certain portion of the field-of-view as opposed to other portions…certain regions of the field-of-view 14 may be defined as having low or no priority. Thus, for example, the user may define a region that should be ignored or the machine-readable symbol reader 12 may determine that a particular region such as the lower left corner of the field-of-view 14 should consistently be ignored based on the history of prior symbol acquisition and/or decoding.”), wherein the method determines whether the visual feature is within the set third ROI (Petrou: 0079: “The system and method may further use the location of objects in different frames to determine whether the objects are the same or different items…FIGURE 15 superimposes the bounding boxes of the three pairs of visually similar objects of frames 1411 and 1421 relative to the edges of the frames. All of the regions defined by the bounding box overlap. Accordingly, the processor may determine that the objects are likely associated with the same item, i.e., the same bottle instead of two different bottles.”).
Petrou in view of Ray and Al Rashdan does not disclose expressly: if the visual feature is determined not to be within the set third ROI, updating the third ROI to be a lower ranked ROI of the first ROI and the second ROI.
Ray further discloses: if the visual feature is determined not to be within the set third ROI, updating the third ROI to be a lower ranked ROI of the first ROI and the second ROI (Ray: 0087: “The user may select how quickly the first switch must be activated following the display of an indicator to cause the machine-readable symbol reader to select a next highest ranked target rather than automatically processing the currently highest ranked target. ”;
0097: “If the user input indicates that a next target should be selected, the machine-readable symbol reader 12 increments the target rank at 136 such that the next highest ranked target becomes active, and returns control to 120 to project or otherwise transmit one or more corresponding indicators to the user. In this way the user can increment through a number of targets in a ranked, sorted or prioritized order with the simple activation of the second switch 82 and/or first switch 80.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the iteration through the potential target order based on a user’s skip target processing selection as further taught by Ray for the processing of an ROI based on the ranked ROI list disclosed by Petrou in view of Ray and Al Rashdan by iterating to a next ROI based on a determination of the object not being within the ROI. The suggestion/motivation for doing so would have been “machine-readable symbol reader 12 may rank, sort or order targets based on a comparison of certain characteristics of the potential target against one or more criteria…If the machine-readable symbol reader 12 determines at 126 that the time has been exceeded, in one embodiment that indicates that the user desires the currently highest ranked target to be processed…If the user input indicates that a next target should be selected, the machine-readable symbol reader 12 increments the target rank at 136 such that the next highest ranked target becomes active, and returns control to 120 to project or otherwise transmit one or more corresponding indicators to the user.” (Ray: 0095-0099; Wherein the rank incrementation allows for the processing of targets based on a best to worst order.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Petrou in view of Ray and Al Rashdan with the further teaching disclosed by Ray to obtain the invention as specified in claim 2.
Regarding claim 3, Petrou in view of Ray and Al Rashdan discloses: The method of claim 2, wherein the analyzing the third ROI further includes: if the visual feature is determined not to be within the updated third ROI, setting the third ROI to be a FOV of the third image (Petrou: 0089: “By way of example only, a Lucas-Kanade pyramidal optical flow method may be used to track feature correspondence between images. Coarse-to-fine tracking may be performed by iteratively adjusting the alignment of image patches around the points from image to image, starting with the smallest, coarsest pyramid level and ending with the finest pyramid level. The feature correspondences may be stored in a circular buffer for a certain period of time such as a number of seconds. This may allow the processor to replay the flow information in order to align features from an earlier image, which may be annotated, with their position within the latest image. An initial estimate of the point-wise delta may be computed by using the two frames to generate a full-frame transformation matrix that describes the translation and rotation that was likely applied to the device between the two frames. ”; Wherein the optical flow methods, processing a current frame and a previous frame, used to track features in the current frame constitutes setting the ROI to an FOV.).
Regarding claim 4, Petrou in view of Ray and Al Rashdan discloses: The method of claim 3, wherein the analyzing the third ROI further includes: analyzing the third ROI that has been set to the FOV of the third image to detect the visual feature; determining, via the application, a location of the visual feature within the third image; determining, via the application, a new third ROI within the third image based on the location of the visual feature (Petrou: Figure 16: t0 & t1;
0089: “a Lucas-Kanade pyramidal optical flow method may be used to track feature correspondence between images. Coarse-to-fine tracking may be performed by iteratively adjusting the alignment of image patches around the points from image to image, starting with the smallest, coarsest pyramid level and ending with the finest pyramid level…This may allow the processor to replay the flow information in order to align features from an earlier image, which may be annotated, with their position within the latest image…The resulting point is where the original point would be located if it followed the overall transformation between frames…Optical flow may be subject to drift in which case relocalization may be used and, if the relocalization fails, tracking of the object may be stopped until the object is reacquired.”; Wherein the optical flow algorithm is used to track an item/feature from its position in a previous frame to its position in the current frame.); determining, via the application, that the new third ROI is within a predetermined tolerance of the first ROI; and in response to the determination that the new third ROI is within a predetermined tolerance of the first ROI, incrementing, via the application, a weighting factor of the first ROI (Petrou: 0070: “The system and method may also weigh information obtained from the most recent frames more heavily than information obtained from older frames. For instance, when preparing a query based on the frequency of descriptions across three of the most recent frames, the processor may give an object a relative weight of 1.00 if the object only appears in the most recent frame, a weight of 0.25 if the object only appears in the oldest frame, and a weight of 1.75 (equal to 1.00+0.50+0.25) if the object appears in all three frames. The system and method may determine and weigh other signals than those described herein.”;
0089: “Once objects are identified, they may have positions and scales tracked and updated from frame to frame, at a rate between 15 and 30 frames/second, according to the features that fall within or around a bounding box created for the object…Optical flow may be subject to drift in which case relocalization may be used and, if the relocalization fails, tracking of the object may be stopped until the object is reacquired.”).
Regarding claim 5, Petrou in view of Ray and Al Rashdan discloses: The method of claim 1, further comprising: subsequent to the analyzing of the ROI of the third image, iteratively capturing images (Petrou: 0006: “ the device may simultaneously display two or more of the following: (a) the image sent to the server, (b) an image visually similar to the image sent to the server, such as a subsequent frame of a video stream”), and, at each iteration: incrementing a weighting factor of the first ROI if the visual feature is determined to be within the first ROI; incrementing a weighting factor of the second ROI if the visual feature is determined to be within the second ROI; and re-ranking the first and second ROIs based on the weighting factors (Petrou: 0070: “The system and method may also weigh information obtained from the most recent frames more heavily than information obtained from older frames. For instance, when preparing a query based on the frequency of descriptions across three of the most recent frames, the processor may give an object a relative weight of 1.00 if the object only appears in the most recent frame, a weight of 0.25 if the object only appears in the oldest frame, and a weight of 1.75 (equal to 1.00+0.50+0.25) if the object appears in all three frames. The system and method may determine and weigh other signals than those described herein.”;
Wherein each feature detected in the ROIs in each frame is weighted and thus ranked based on the frequency of detection of their respective specific feature in the ROI.).
Regarding claim 6, Petrou in view of Ray and Al Rashdan discloses: The method of claim 1, wherein: the analyzing the at a portion of the first image to detect the visual feature comprises determining, via the application, a bounding box of the visual feature (Petrou: Figure 14; 0079: “FIGURE 14 illustrates a sequence of frames 1411 and 1421. The processor detects three shapes in the first image, namely, the bottle shape, logo and bar code. The processor further determines a bounding box 1412-14 for each shape.”); and the determining the first ROI within the first image comprises applying a scaling factor to the bounding box (Al Rashdan: 0070: “a model (e.g., of DL module 608 of FIG. 6 ) may be run on the converted image object (i.e., to detect one or more codes (e.g., QR codes)), and method 700 may proceed to block 706 . At block 706, it may be determined whether one or more codes are detected. If the model was able to detect a code, a bounding box may be generated (e.g., at least partially around the detected code), and method 700 may proceed from block 706 to block 708, where a bounding box region is expanded (e.g., to improve detectability). For example, a bounding box region may be expanded (e.g., by a percentage p, such as between 5% and 20%) to ensure edges of the code do not fall outside an original bounding box region.”).
Regarding claim 7, Petrou in view of Ray and Al Rashdan discloses: The method of claim 1, further comprising: responsive to detecting the visual feature in the first image, transmitting data associated with the visual feature in the first image to the host processor; and responsive to detecting the visual feature in the second image, transmitting data associated with the visual feature in the second image to the host processor (Petrou: Figures 9 & 10; 0072: “A processor may select a subset of the returned results and display the selected subset to the user. This may include selecting the highest ranking result as the optimum annotation.”; Wherein the higher ranking features detected in the first and second images are processed and selected by the processor.).
Regarding claim 8, Petrou in view of Ray and Al Rashdan discloses: The method of claim 1.
Petrou in view of Ray and Al Rashdan does not disclose expressly: wherein the ranking the first and second ROIs comprises: presenting, to a user, a representation of the first ROI and the second ROI; receiving, from the user, a user selection of either the first ROI or the second ROI; and setting the ranks of the first ROI and the second ROI based on the user selection.
Ray further discloses: the ranking of ROIs detected in the image based on user configurable preferences, such as the user being able to select regions/ROI present in the field of view of the image to ignore, thus removing them from detection (Ray: 0109: “the processor 42 may rank, sort, order or prioritize regions of interest based on a predefined criteria or preference, a user defined preference, or criteria or preference determined from a historical pattern of usage of the machine-readable symbol reader 12 and/or user.”; 0112: “the user may define a region that should be ignored or the machine-readable symbol reader 12 may determine that a particular region such as the lower left corner of the field-of-view 14 should consistently be ignored based on the history of prior symbol acquisition and/or decoding.”).
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to implement the known technique as further taught by Ray of ranking features and ROIs present in images based on user preferences into the ranking of features disclosed by Petrou in view of Ray and Al Rashdan. The suggestion/motivation for doing so would have been “the criteria or preference may rank small machine-readable symbols higher, for example where all machine-readable symbols appear on a single plane such as a standardized form, and the user recognizes that the desired information is consistently encoded in the smallest of the machine-readable symbols” (Ray: 0111; Wherein the user is able to make selections regarding the information/regions to prioritize based on their own observations.). Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine Petrou in view of Ray and Al Rashdan with the further teaching of Ray to obtain the invention as specified in claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANTHONY J RODRIGUEZ whose telephone number is (703)756-5821. The examiner can normally be reached Monday-Friday 10am-7pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANTHONY J RODRIGUEZ/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights