DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 10 July 2026 have been fully considered but they are not persuasive.
Applicant merely asserts that the applied art does not disclose the amended claim language of “applying an additional set of data labels to the subset of images, wherein the additional set of data labels include at least one of a revised bounding box surrounding one of the first set of objects or identifying an additional object not identified in the first set of data labels.”
No substantive rationale, logic, citation, or argument is provided to support this assertion.
Therefore, no substantive counter-argument is due other than the counter-assertion set forth in the revised prior art rejections that address the amended claim language.
It is noted, however, that Applicant has not mentioned, let alone challenged any of the findings made in the first office action on the merits which included a detailed claim interpretation under BRI (broadest reasonable interpretation) of various claim elements including “identifying an object” which is curiously unilluminating of Applicant’s position particularly because the amended claim language refers to “identifying an additional object not identified in the first set of data labels”.
For ease of reference the first action and this action finds that “identifying an object” is being used quite broadly and has a BRI that includes object detection that detects but does not recognize the categorical identity of an object. Indeed, [0030] and [0040] of the published instant specification employs the terms “identifying” and “detecting” synonymously. Moreover, the instant specification does not provide adequate written support for the more specific object recognition that the term “identifying” may imply because there is no description of identifying/recognizing, for example a bicycle or pedestrian object category/type or otherwise distinguishing between object types such that the most reasonable interpretation of “identifying an object” and “identifying an additional object” consistent with the specification is object detection that detects presence of an object but does not classify the object or otherwise recognize the object.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 1-4, 11-15, and 19-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Contrary to MPEP 2163(II)(A) and 21603.04(I)(B), no disclosure has been identified by Applicant in their 10 July 2026 Reply pointing out where the amended claim language added to any of the amended claims is supported by the application as filed.
Moreover, the Examiner could not find any disclosure as filed supporting the concepts now recited in amended claims 2 and 14.
Claim 2 has been amended to recite “the first set of data labels includes a first bounding box and object type for each of the first set of objects, and the revised bounding box includes a bounding box that more closely surrounds a corresponding one of the first set of objects as compared to the first bounding box from the first set of data labels.” Claim 14 has been similarly amended.
Indeed, the first office action on the merits found that “the instant specification does not provide adequate written support for the more specific object recognition that the term “identifying” may imply because there is no description of identifying/recognizing, for example a bicycle or pedestrian object category/type or otherwise distinguishing between object types such that the most reasonable interpretation consistent with the specification is object detection that detects presence of an object.”
Applicant has chosen to ignore this finding and has not presented any counter-arguments in their Reply. Instead, the problem has been magnified by now impliedly claiming identifying object type and specifically applying data labels with the object type which presupposes and requires object recognition/classification in order to provide object type labels. Furthermore, the instant specification does not even include the word “type” a single time in the entire specification let alone the concept of recognizing object type. Instead, the instant specification is limited to object detection that detects objects and their bounding boxes but does not recognize, classify or otherwise determine object type.
Further as to the “bounding box that more closely surrounds a corresponding one of the first set of objects as compared to the first bounding box from the first set of data labels” the only mention of this feature is in [0034] which parrots the claim language without providing any details as to how the bounding box that more closely surrounds the object. Still further, it is not clear if this feature is manually or automatically performed by an undisclosed method.
Independent claim 1 has also been amended to recite “identifying an additional object not identified in the first set of data labels”. Similar amendments are made to claims 3, 11, 13, 14, 19, and 20. Again, [0034] of the instant spec appears to be the sole source of disclosure on this feature but does not explain how unidentified objects are now being identified. It is also unclear whether this step is being performed manually or with some undisclosed automatic method.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-4, 11-15, and 19-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Amended claim 1 recites “identifying an additional object not identified in the first set of data labels”. Similar amendments are made to claims 3, 11, 13, 14, 19, and 20. This claim element lacks context as there is no determination of objects not identified in the first set of data labels.
Claims 2, 4, 12, 15, and 19-20 are rejected due to their dependency.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 3, 11-13, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 2023/0202525 A1) and Chang (US 2023/0259199 A1).
Claim 1
In regards to claim 1, Wu discloses a method of analyzing an image dataset {See abstract, Fig. 1 adaptive situational awareness support application 106, Fig. 5 method and cites below}, the method comprising:
obtaining a dataset having a plurality of images of an area surrounding a vehicle
{Fig. 6, receive image data associated with driving scene of ego vehicle step 602, [0102]. For hardware, see Fig. 1 including vehicle camera system 110 obtaining the claimed dataset of images. See also Figs. 2-3.};
identifying a first set of objects in each image of the plurality of images and applying a first set of data labels to the first set of objects in the plurality of images with a trained object detector describing each of the first set of objects
{initially, it is noted that the term “identifying an object” is being used quite broadly has a BRI that corresponds to object detection that detects but does not recognize the categorical identity or type of an object. Indeed, [0030] and [0040] of the published instant specification employs the terms “identifying” and “detecting” synonymously. Moreover, the instant specification does not provide adequate written support for the more specific object recognition that the term “identifying” may imply because there is no description of identifying/recognizing, for example a bicycle or pedestrian object category/type or otherwise distinguishing between object types such that the most reasonable interpretation consistent with the specification is object detection that detects presence of an object.
See Figs. 4, 6 including driving scene determinant module 122 and analyzing/detecting objects located in the driving scene step 404, 604, [0061]-[0066] that detects static and dynamic objects and computes/extracts bounding boxes around each of the objects 202, 204 as also shown in Figs. 2-3. Further as to applying a first set of data labels, the object classification uses a pre-trained object classifier that applies detection and classification labels to static and dynamic objects 204, 202. See also the bounding boxes that module 122 determines for each object wherein the bounding boxes are also themselves the first set of data labels describing each of the first set of objects};
obtaining eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze
{Fig. 1 including eye gaze sensors 112, Fig. 4, receiving eye gaze data step 408; [0027], [0053]-[0055] which extracts gaze coordinates and employs a homography between the driving scene 200 and eye gaze scene to determine correspondence and matching of gaze coordinates and object coordinates for each of the images. Fig. 5, eye fixation detection step 502, [0070]-[0074}; and
identifying a subset of images from the plurality of images based on a relationship between the eye-gaze
identified in each image of the plurality of images and applying an additional set of data labels to the subset of images, wherein the additional set of data labels include at least one of a revised bounding box surrounding one of the first set of objects or identifying an additional object not identified in the first set of data labels.
{initially, it is noted that the BRI of this claim element encompasses a subset of whole (entire) images from among the plurality of entire images and portions/objects of interest/regions of interest within an entire image as per [0040]-[0042] of the instant application.
See Fig. 5 including determining a level of situational awareness step 504 which includes identifying a subset of images (objects) to which the eye-gaze information indicates fixations/saccades of those objects and performing additional data labeling (labeling each of the dynamic objects with fixation and/or saccade labels) as well as further/additional labeling a subset of objects with a level of situational awareness. Still additional labeling in the form of an object classification label is assigned to a subset of object’s region of interest exceeding a threshold level of high situational awareness (e.g. 8-10) , [0075]-[0080]
See also Fig. 6 analyze eye gaze data step 604 and determining level of importance of objects, [0102]-[0103]. See also [0028], [0036], [0053]. Further as to “subset of images” and “applying an additional set of data labels to the subset of images” note that a) detected objects are subjected to additional data labeling such as dynamic, static, fixation, saccade, situational awareness, and classification data labeling while b) additional objects not identified in the first set of labels including background objects, non-object areas and scene complexity are analyzed in block 406 to provide additional data labelling such as recognized roadway configurations, sidewalk configurations, terrain of the driving scene and/or environmental conditions (e.g., weather conditions, road conditions)}.
Although Wu discloses obtaining eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determining a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images, Wu is not relied upon to disclose using eye-gaze direction information for these purposes.
Chang is analogous art because it reasonably pertinent to the problem faced by the inventor which is finding the correspondence between gaze information and real-world objects being viewed by an operator/human. See abstract, Figs. 1-3, 6 and cites below.
Chang also teaches
obtaining eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze direction of the operator corresponding to each of the plurality of images and determining a relationship between the eye-gaze direction to a a position of the at least one object identified in each image of the plurality of images {See Fig. 2, [0031]-[0035], Fig. 5, [0041]-[0047]; [Fig. 6 including identifying targets/objects in an image S610, tracking gaze direction step S615 and identifying a subset of targets based on gaze direction for further processing S620, [0048]-[0053]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location to a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 3
In regards to claim 3, Wu discloses wherein the relationship between the eye-gaze
applying the additional set of data labels to the subset of images includes applying a data label to an object not identified by the first set of data labels.
{see claim 1 mapping while noting that a) detected objects are subjected to additional data labeling such as dynamic, static, fixation, saccade, situational awareness, and classification data labeling while b) additional objects not identified in the first set of labels including background objects, non-object areas and scene complexity are analyzed in block 406 to provide additional data labelling such as recognized roadway configurations, sidewalk configurations, terrain of the driving scene and/or environmental conditions (e.g., weather conditions, road conditions}.
Chang also teaches wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object {See Fig. 2, [0031]-[0035], Fig. 5, [0041]-[0047]; [Fig. 6 including identifying targets/objects in an image S610, tracking gaze direction step S615 and identifying a subset of targets based on gaze direction for further processing S620, [0048]-[0053]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang and such that wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object as also taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 11
In regards to claim 11, Wu discloses wherein the first set of objects is offset by a predetermined angular range from a heading of the vehicle and the first set of objects includes a speed within a predetermined range
{it is noted that this claim merely recites properties of an object (offset by angular range, object speed) and properties of a vehicle (heading of vehicle). It is further noted that a typical driving scene such as that captured by Wu includes various objects some of which will inherently have a heading that is offset from the vehicle’s heading by a predetermined range. Moreover, some objects such as other vehicles have speeds within a predetermined range (e.g. static, moving objects such as the static and moving objects discussed by Wu). Furthermore, this claim does not require or recite determining any of these properties. In other words, the claim does NOT recite determining vehicle heading, object speed or offset by a predetermined angular range. Indeed, the specification is extremely short on adequate written description support for any such determinations and positively reciting such determinations would very likely result in a rejection under 35 USC 112(a). In sum, this claim merely describes the operating environment of a vehicle travelling down a road with cameras observing objects that have these properties relative to the vehicle and which are clearly met by Wu’s identical operating environment and typical driving scenarios that includes myriad objects travelling at different headings, angular offsets and speeds};
applying the additional set of data labels to the subset of images includes applying a data label to an object not within the first set of data labels.
{see claim 1 mapping while noting that a) detected objects are subjected to additional data labeling such as dynamic, static, fixation, saccade, situational awareness, and classification data labeling while b) additional objects not identified (within) in the first set of labels including background objects, non-object areas and scene complexity are analyzed in block 406 to provide additional data labelling such as recognized roadway configurations, sidewalk configurations, terrain of the driving scene and/or environmental conditions (e.g., weather conditions, road conditions}.
Claim 12
In regards to claim 12, Wu discloses applying the additional set of data labels to the set to the subset of images includes applying the additional set of data labels to a group of objects identified in a region of interest based on the eye-gaze
{see above claim interpretation for claim 11 which also applies to claim 12. Again, Applicant claims mere properties of an object and vehicle (heading in a direction transverse to vehicle heading). This claim does not require or recite determining any of these properties. In other words, the claim does NOT recite determining vehicle heading or object heading. Nor does this claim recite any determination of object heading transverse to vehicle heading. Indeed, the specification is extremely short on adequate written description support for any such determinations and positively reciting such determinations would very likely result in a rejection under 35 USC 112(a). It is further noted that a typical driving scene such as that captured by Wu includes various objects some of which will inherently have headings transverse to the vehicle heading}.
Chang also teaches
obtaining eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze direction of the operator corresponding to each of the plurality of images and determining a relationship between the eye-gaze direction to a a position of the at least one object identified in each image of the plurality of images {See Fig. 2, [0031]-[0035], Fig. 5, [0041]-[0047]; [Fig. 6 including identifying targets/objects in an image S610, tracking gaze direction step S615 and identifying a subset of targets based on gaze direction for further processing S620, [0048]-[0053]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location to a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 13; 19 and 20
The rejection of method claims 1; 1 and 3 above applies mutatis mutandis to the corresponding limitations of computer readable storage medium claim 13; and system claims 19 and 20 while noting that the rejection above cites to both device and method disclosures. For the computer readable storage medium storing program limitations of claim 13 see [0104]-[0105].
Further as to claim 19-20 recitation of the at least one distance sensor configured to measure a plurality of distances from the at least one distance sensor; and a controller in communication with the at least one optical sensor, the at least one distance sensor, see Chang [0005], [0022], [0023], [0025], [0027], [0035], [0043]-[0048], Fig. 7 processor 705, memory 710.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang and such that wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object as also taught by Chang and such that
wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the projection of the eye-gaze direction relative to the position of the at least one object includes the projection of the eye-gaze direction intersecting the at least one object as further taught by Chang and such that the system includes at least one distance sensor configured to measure a plurality of distances from the at least one distance sensor; and a controller in communication with the at least one optical sensor, the at least one distance sensor as also taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because motivates including a distance sensor such that the depth map therefrom can be used to estimate gaze direction and select subset of targets as per [0043] to improve estimating targets that intersect the gaze line as per [0044], because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 2 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Wu and Chang as applied to claims 1/13 above, and further in view of Upsenyava {Uspenyeva, Lisa “Best Bounding Box Image Annotation Tools For Object Detection - Complete Overview” https://supervisely.com/blog/bounding-box-annotation-for-object-detection/ downloaded 1 September 2026, published 20 October 2023}.
Claim 2
In regards to claim 2, Wu discloses wherein the plurality of images are obtained from at least one optical sensor
the first set of data labels includes a first bounding box and object type for each of the first set of objects {see mapping for claim 1 including See Figs. 4, 6 including driving scene determinant module 122 and analyzing/detecting objects located in the driving scene step 404, 604, [0061]-[0066] that detects static and dynamic objects and computes/extracts bounding boxes around each of the objects 202, 204 as also shown in Figs. 2-3. Further as to applying a first set of data labels, the object classification uses a pre-trained object classifier that applies detection and classification labels to static and dynamic objects 204, 202. See also the bounding boxes that module 122 determines for each object wherein the bounding boxes are also themselves the first set of data labels describing each of the first set of objects}, and
Chang also teaches that a distance sensor that is used in conjunction with a camera to estimate daze direction relative to various objects. See [0043]-[0048].
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location to a position of the at least one object identified in each image of the plurality of images such that the method/system includes a distance sensor as taught by Chang because depth maps created from the distance sensor can be used to increase the accuracy of identifying targets that are in the near-field as motivated by Chang in [0043]-[0048], because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Upsenyava is a highly relevant and analogous reference from the same field of object detection and labelling of detected objects. Indeed, Upsenyava summarizes the commercially available tool Supervisely that is specifically designed to perform object detection and annotation/labeling of detected objects including application to autonomous vehicles. See pgs. 1-4.
Upsenyava also teaches applying an additional set of data labels to the subset of images, wherein the additional set of data labels includes a revised bounding box surrounding one of the first set of objects, the revised bounding box includes a bounding box that more closely surrounds a corresponding one of the first set of objects as compared to the first bounding box from the first set of data labels. See Auto-Select for fast editing of Bounding Box, pgs. 5-6, which enables a user to revise/edit a bounding box initially drawn by a Neural Network which corresponds to claim 1’s applying a first set of data labels to the first set of objects in the plurality of images with a trained object detector describing each of the first set of objects. Moreover, the video on this website clearly demonstrates that the bounding boxes are manually revised by the user to more closely surround the object as compared with the bounding box drawn by the Neural Network.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already identifies a first set of objects in each image of the plurality of images, applies a first set of data labels to the first set of objects in the plurality of images with a trained object detector describing each of the first set of objects, and applies an additional set of data labels to the subset of images such that the additional set of data labels includes a revised bounding box surrounding one of the first set of objects, the revised bounding box includes a bounding box that more closely surrounds a corresponding one of the first set of objects as compared to the first bounding box from the first set of data labels as taught by Upsenyava because doing so increases the accuracy of the bounding boxes while reducing the burden for such manual revision by applying a first set of data labels including the bounding box by a trained object detector and then manually revising/refining the bounding boxes, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 14
The rejection of method claims 2 and 3 above applies mutatis mutandis to the corresponding limitations of computer readable storage medium claim 14 while noting that the rejection above cites to both device and method disclosures. For the computer readable storage medium storing program limitations of claim 14 see [0104]-[0105].
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Wu and Chang as applied to claims 3 above, and further in view of Xia {Xia, Ye, et al. "Predicting Driver Attention in Critical Situations.", UC Berkeley Previously Published Works, https://escholarship.org/uc/item/0vx1h4j4, DOI 10.1007/978-3-030-20873-8_42 (2019)}
Claim 4
In regards to claim 4, Wu discloses wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the
Xia is an analogous reference from the same field of object detection including the disclosed but unclaimed application to driving applications. See abstract, Introduction and cites below.
Xia also teaches that the method includes training a neural network with the subset of images to perform object detection {see abstract, sections 1 3, 4 and 5,Fig. 1, which uses a gaze map (attention map) to identify a subset of images for training a neural network to perform object detection.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already identifies a first set of objects in each image of the plurality of images, applies a first set of data labels to the first set of objects in the plurality of images with a trained object detector describing each of the first set of objects, and applies an additional set of data labels to the subset of images such that the method/system includes training a neural network with the subset of images to perform object detection as taught by Xia because Xia motivate using such a subset as opposed to a large dataset of unselected data to increase accuracy of the model in section 1, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Chang also teaches wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the projection of the eye-gaze direction relative to the position of the at least one object includes the projection of the eye-gaze direction intersecting the at least one object
{see above cites for claim 3. Further as to eye-gaze direction intersecting the object see [0005], [0035]- [0053]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang and such that wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object as also taught by Chang and such that
wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the projection of the eye-gaze direction relative to the position of the at least one object includes the projection of the eye-gaze direction intersecting the at least one object as further taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 15
The rejection of method claim 4 above applies mutatis mutandis to the corresponding limitations of computer readable storage medium claim 15 while noting that the rejection above cites to both device and method disclosures. For the computer readable storage medium storing program limitations of claim 15 see [0104]-[0105].
Claims 5-10 and 16-18: Withdrawn-Non-elected
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
S. Vishwakarma, D. Radha and J. Amudha, "Effectual Training for Object Detection Using Eye Tracking Data Set," 2018 International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2018, pp. 225-230, doi: 10.1109/ICIRCA.2018.8597275 is also highly relevant and discloses eye tracking data to determine a fixation map for objections in which a subset of detected objects is subjected to additional labeling based on the eye fixation points/duration.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/Primary Examiner, Art Unit 2667
Prosecution Insights