DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 12 March 2026 have been fully considered but they are not persuasive.
Applicant traverses the restriction requirement by arguing that the action does not identify distinct search classes or explain how the groupings are divergent and that “every case involves a search of multiple classes/subclasses and multiple search queriers. As such, Applicant alleges no serious search burden has been established.”
Although employing separate classifications is one way to establish serious search burden, the action uses another, valid way which is to set forth serious search burden via the overly burdensome search strategies and fields demanded by the diverse subject matter of the inventive groups. This rationale includes a wide range of search strategies and fields of search for finding art relative to the proximity of a projection of the eye-gaze direction relative to or intersecting the position of the at least one object which wholly differs from search strategies and fields for finding art relative to a selected image from the plurality of images being included in the subset of images by determining if the proximity of the projection of the eye-gaze direction relative to the position of the at least one object is within a predetermined range. Still further the search strategy and fields for the various sub-species substantially differ from one another including diverse subjects of angular range, distance range, projection intersections, and frequency of projections which all contribute to an unreasonable search burden should each of these diverse subjects and their diverse search terms be examined together.
Upon review of the action and arguments, the evidence demonstrates establishment of a serious search burden sufficient to maintain the restriction requirement. As such, examination will proceed with elected Species A having claims 1-4, 11-15 and 19-20 with claims 5-10 and 16-18 being withdrawn as directed to a non-elected invention(s).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-4, 11-15 and 19-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
All of the independent claims 1, 13 and 19 recite “identifying a subset of images from the plurality of images for performing additional data labeling”. The term “additional” data labeling is unclear because it lacks a frame of reference. In other words, in addition to what (unspecified, unrecited) labeling does this “additional” data labeling refer? Claims 2-4, 11-12, 14, 15 and 20 are rejected due to their dependency.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 11-15 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Wu (US 2023/0202525 A1) and Chang (US 2023/0259199 A1).
Claim 1
In regards to claim 1, Wu discloses a method of analyzing an image dataset {See abstract, Fig. 1 adaptive situational awareness support application 106, Fig. 5 method and cites below}, the method comprising:
obtaining a dataset having a plurality of images of an area surrounding a vehicle
{Fig. 6, receive image data associated with driving scene of ego vehicle step 602, [0102]. For hardware, see Fig. 1 including vehicle camera system 110 obtaining the claimed dataset of images. See also Figs. 2-3.};
identifying at least one object in each image of the plurality of images
{initially, it is noted that the term “identifying an object” is being used quite broadly has a BRI that includes object detection that detects but does not recognize the categorical identity of an object. Indeed, [0030] and [0040] of the published instant specification employs the terms “identifying” and “detecting” synonymously. Moreover, the instant specification does not provide adequate written support for the more specific object recognition that the term “identifying” may imply because there is no description of identifying/recognizing, for example a bicycle or pedestrian object category/type or otherwise distinguishing between object types such that the most reasonable interpretation consistent with the specification is object detection that detects presence of an object. See Figs. 4, 6 including driving scene determinant module 122 and analyzing/detecting objects located in the driving scene step 404, 604, [0061]-[0066] that detects static and dynamic objects and computes/extracts bounding boxes around each of the objects 202, 204 as also shown in Figs. 2-3};
obtaining eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze
{Fig. 1 including eye gaze sensors 112, Fig. 4, receiving eye gaze data step 408; [0027], [0053]-[0055] which extracts gaze coordinates and employs a homography between the driving scene 200 and eye gaze scene to determine correspondence and matching of gaze coordinates and object coordinates for each of the images. Fig. 5, eye fixation detection step 502, [0070]-[0074}; and
identifying a subset of images from the plurality of images for performing additional data labeling based on a relationship between the eye-gaze
{initially, it is noted that the BRI of this claim element encompasses a subset of whole (entire) images from among the plurality of entire images and portions/objects of interest/regions of interest within an entire image as per [0040]-[0042] of the instant application.
See Fig. 5 including determining a level of situational awareness step 504 which includes identifying a subset of images (objects) to which the eye-gaze information indicates fixations/saccades of those objects and performing additional data labeling (labeling each of the dynamic objects with fixation and/or saccade labels) and also further labeling a subset of objects with a level of situational awareness. Still additional labeling in the form of an object classification label is assigned to a subset of object’s region of interest exceeding a threshold level of high situational awareness (e.g. 8-10) , [0075]-[0080]
See also Fig. 6 analyze eye gaze data step 604 and determining level of importance of objects, [0102]-[0103]. See also [0028], [0036], [0053]. Further as to “subset of images” note only the detected objects are subjected to additional data labeling while other portions of the image (non-object areas, background, etc.) are not labelled based on a relationship between the eye-gaze information and position of object(s) detected in the images}.
Although Wu discloses obtaining eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determining a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images, Wu is not relied upon to disclose using eye-gaze direction information for these purposes.
Chang is analogous art because it reasonably pertinent to the problem faced by the inventor which is finding the correspondence between gaze information and real-world objects being viewed by an operator/human. See abstract, Figs. 1-3, 6 and cites below.
Chang also teaches
obtaining eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze direction of the operator corresponding to each of the plurality of images and determining a relationship between the eye-gaze direction and a position of the at least one object identified in each image of the plurality of images {See Fig. 2, [0031]-[0035], Fig. 5, [0041]-[0047]; [Fig. 6 including identifying targets/objects in an image S610, tracking gaze direction step S615 and identifying a subset of targets based on gaze direction for further processing S620, [0048]-[0053]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 2
In regards to claim 2, Wu discloses wherein the plurality of images are obtained from at least one optical sensor on the vehicle when the vehicle is operated in a non-autonomous mode {see above mapping of claim 1 including Fig. 1 vehicle camera system 110 obtaining the claimed dataset of images. See also Figs. 2-3. These images are collected from a live person operating a motor vehicle in a non-autonomous mode as per [0027]. [0053], [0069]-[0082]}.
Claim 3
In regards to claim 3, Wu discloses wherein the relationship between the eye-gaze
Chang also teaches wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object {See Fig. 2, [0031]-[0035], Fig. 5, [0041]-[0047]; [Fig. 6 including identifying targets/objects in an image S610, tracking gaze direction step S615 and identifying a subset of targets based on gaze direction for further processing S620, [0048]-[0053]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang and such that wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object as also taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 4
In regards to claim 4, Wu discloses wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the
Chang also teaches wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the projection of the eye-gaze direction relative to the position of the at least one object includes the projection of the eye-gaze direction intersecting the at least one object
{see above cites for claim 3. Further as to eye-gaze direction intersecting the object see [0005], [0035]- [0053]}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang and such that wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object as also taught by Chang and such that
wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the projection of the eye-gaze direction relative to the position of the at least one object includes the projection of the eye-gaze direction intersecting the at least one object as further taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 5-10 and 16-18: Withdrawn-Non-elected
Claim 11
In regards to claim 11, Wu discloses wherein the at least one object is offset by a predetermined angular range from a heading of the vehicle and the at least one object includes a speed within a predetermined range {it is noted that this claim merely recites properties of an object (offset by angular range, object speed) and properties of a vehicle (heading of vehicle). It is further noted that a typical driving scene such as that captured by Wu includes various objects some of which will inherently have a heading that is offset from the vehicle’s heading by a predetermined range. Moreover, some objects such as other vehicles have speeds within a predetermined range (e.g. static, moving objects such as the static and moving objects discussed by Wu). Furthermore, this claim does not require or recite determining any of these properties. In other words, the claim does NOT recite determining vehicle heading, object speed or offset by a predetermined angular range. Indeed, the specification is extremely short on adequate written description support for any such determinations and positively reciting such determinations would very likely result in a rejection under 35 USC 112(a). In sum, this claim merely describes the operating environment of a vehicle travelling down a road with cameras observing objects that have these properties relative to the vehicle and which are clearly met by Wu’s identical operating environment and typical driving scenarios that includes myriad objects travelling at different headings, angular offsets and speeds}.
Claim 12
In regards to claim 12, Wu discloses wherein the at least one object in the subset of images includes a heading in a direction transverse to a heading of the vehicle {see above claim interpretation for claim 11 which also applies to claim 12. Again, Applicant claims mere properties of an object and vehicle (heading in a direction transverse to vehicle heading). This claim does not require or recite determining any of these properties. In other words, the claim does NOT recite determining vehicle heading or object heading. Nor does this claim recite any determination of object heading transverse to vehicle heading. Indeed, the specification is extremely short on adequate written description support for any such determinations and positively reciting such determinations would very likely result in a rejection under 35 USC 112(a). It is further noted that a typical driving scene such as that captured by Wu includes various objects some of which will inherently have headings transverse to the vehicle heading}.
Claims 13, 14, 15; 19 and 20
The rejection of method claims 1, 3, 4; 1 and 3 above applies mutatis mutandis to the corresponding limitations of computer readable storage medium claims 12, 14 and 15; and system claims 19 and 20 while noting that the rejection above cites to both device and method disclosures. For the computer readable storage medium storing program limitations of claims 13 and 14 see [0104]-[0105].
Further as to claim 19-20 recitation of the at least one distance sensor configured to measure a plurality of distances from the at least one distance sensor; and a controller in communication with the at least one optical sensor, the at least one distance sensor, see Chang [0005], [0022], [0023], [0025], [0027], [0035], [0043]-[0048], Fig. 7 processor 705, memory 710.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Wu which already obtains eye-gaze information directed to an operator of the vehicle from an eye-gaze monitoring system, wherein the eye-gaze information includes an eye-gaze locations of the operator corresponding to each of the plurality of images and determines a relationship between the eye-gaze location and a position of the at least one object identified in each image of the plurality of images such that eye-gaze direction information is used for these purposes as taught by Chang and such that wherein the relationship between the eye-gaze direction and the position of the at least one object is based on a proximity of a projection of the eye-gaze direction relative to the position of the at least one object as also taught by Chang and such that
wherein a selected image from the plurality of images is included in the subset of images by determining if the proximity of the projection of the eye-gaze direction relative to the position of the at least one object includes the projection of the eye-gaze direction intersecting the at least one object as further taught by Chang and such that the system includes at least one distance sensor configured to measure a plurality of distances from the at least one distance sensor; and a controller in communication with the at least one optical sensor, the at least one distance sensor as also taught by Chang because eye gaze direction is a functional equivalent of eye gaze position for determining which objects the person is gazing upon to select objects for further processing or further attention such as labeling, because motivates including a distance sensor such that the depth map therefrom can be used to estimate gaze direction and select subset of targets as per [0043] to improve estimating targets that intersect the gaze line as per [0044], because there is a reasonable expectation of success and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Xia {Xia, Ye, et al. "Predicting Driver Attention in Critical Situations.", UC Berkeley Previously Published Works, https://escholarship.org/uc/item/0vx1h4j4, DOI 10.1007/978-3-030-20873-8_42} is a highly relevant disclosure that leverages eye tracking data for vehicle driving videos to determine a gaze map/attention map to determine sampling weights for training.
S. Vishwakarma, D. Radha and J. Amudha, "Effectual Training for Object Detection Using Eye Tracking Data Set," 2018 International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2018, pp. 225-230, doi: 10.1109/ICIRCA.2018.8597275 is also highly relevant and discloses eye tracking data to determine a fixation map for objections in which a subset of detected objects is subjected to additional labeling based on the eye fixation points/duration.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/Primary Examiner, Art Unit 2667
Prosecution Insights