DETAILED ACTION
This action is in response to application 19/030,582 filed on 01/17/2025.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
3. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory obviousness-type double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the conflicting application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement.
Effective January 1, 1994, a registered attorney or agent of record may sign a terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 37 CFR 3.73(b).
4. Claims 1-20 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1-20 of U.S. Patent No. 12,244,970 B2.
Furthermore, although the conflicting claims at issue are not identical, they are not patentably distinct from each other because U.S. Patent No.: 12,244,970 B2 claims:
Instant Application: 19/030,582
Note: bold and underlined fonts means same features between instant application and conflicting appl.
Conflicting Application: 18/333,943
→ now US Patent No.: 12,244,970 B2
Claim [1]: A non-transitory computer readable medium storing instructions executable by one or more processors to: determine a camera pose of a first camera of a plurality of scene cameras mounted on a system; determine an attention direction of a user of the system based on sensor data captured by an eye tracking device of the system; and capture an image of a physical environment by the first camera, wherein a field of view of the first camera overlaps the attention direction; and identify a region of interest in the physical environment by projecting the attention direction into the image.
Claim [1]: A non-transitory computer readable medium storing instructions executable by one or more processors to: determine a camera pose of a first camera of a plurality of scene cameras mounted on a system; determine an attention direction of a user of the system based on sensor data captured by a user-facing camera of the system; determine that a field of view of the first camera overlaps the attention direction based on the camera pose and the attention direction; and in response to the field of view overlapping the attention direction: obtain an image captured by the first camera; and identify an object of interest in the obtained image by projecting the attention direction into the obtained image to identify a region of interest within the image.
Claim [8]: A method comprising: determining a camera pose of a first camera of a plurality of scene cameras mounted on a system; determining an attention direction of a user of the system based on sensor data captured by an eye tracking device of the system; and capturing an image of a physical environment by the first camera, wherein a field of view of the first camera overlaps the attention direction; and identifying a region of interest in the physical environment by projecting the attention direction into the image.
Claim [8]: A method comprising: determining a camera pose of a first camera of a plurality of scene cameras mounted on a system, the camera pose; determining an attention direction of a user of the system based on sensor data captured by a user-facing camera of the system; determining that a field of view of the first camera overlaps the attention direction based on the camera pose and the attention direction; and in response to the field of view overlapping the attention direction: obtaining an image captured by the first camera; and identifying an object of interest in the obtained image by projecting the attention direction into the obtained image to identify a region of interest within the image.
Claim [15]: A system comprising: a plurality of scene cameras; an eye tracking device; a display; one or more processors; and one or more computer readable media storing instructions executable by the one or more processors to: determine a camera pose of a first camera of the plurality of scene cameras; determine an attention direction of a user based on sensor data captured by the eye tracking device; and capture an image of a physical environment by a first camera of the plurality of scene cameras, wherein a field of view of the first camera overlaps the attention direction; and identify a region of interest in the physical environment by projecting the attention direction into the image.
Claim [15]: A system comprising: a plurality of scene cameras; a user-facing camera; one or more processors; one or more computer readable media storing instructions executable by the one or more processors to: determine a camera pose of a first camera of the plurality of scene cameras mounted on the system, the camera pose associated with the system; determine an attention direction of a user of the system based on sensor data captured by the user-facing camera of the system; determine that a field of view of the first camera overlaps the attention direction based on the camera pose and the attention direction; and in response to the field of view overlapping the attention direction: obtain an image captured by the first camera; and identify an object of interest in the obtained image by projecting the attention direction into the obtained image to identify a region of interest within the image.
Therefore, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains could recognize the advantage of providing a system and method for determining a user’s attention direction captured by a scene camera for providing an image of a vehicle by modifying Wang’s teachings in the present US Patent No.: 12,244,970 B2 for the purpose of identifying an object of interest in the obtained image by projecting the attention direction into the obtained image to identify a region of interest within the image, thereby improving driver safety and enhancing an image capture of a surrounding environment while a user is driving.
Claim Rejections - 35 USC § 103
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Mimar (“Mimar”) (US Pub. No.: 2014/0139655 A1) in view of Tamaki et al. (“Tamaki”) (US Pub. No.: 2011/0025836 A1), and further in view of Tsukizawa et al. (“Tsukizawa”) (US Pub. No.: 2009/0304232 A1).
In regards to claim [1], [8], and [15], Mimar discloses a non-transitory computer readable medium storing instructions (see fig. 5 and/or fig. 9, e.g., “DRAM or flash memory”) executable by one or more processors (see fig. 5, e.g., “processor”), a method (see fig. 31), and a system (see fig. 6) comprising: a plurality of scene cameras (see fig. 1 and/or fig. 6, e.g., “front camera, driver camera, other side camera”); an eye tracking device (see fig. 24, fig. 33, paragraphs [0208] and [0216]); a display (see fig. 1 and/or fig. 6, e.g., “display”); one or more processors (see fig. 6, i.e. “4 core CPU subsystem”), and one or more non-transitory computer readable media (see fig. 5, i.e. "DRAM Memory”) storing instructions executable by the one or more processors (see fig. 5, i.e. “SoC Processor”) to: determine a camera pose of a first camera (see fig. 6, e.g., “front camera”) of the plurality of scene cameras (see fig. 1 and/or fig. 6, e.g., “front camera, driver camera, other side camera”); determine an attention direction of a user based on sensor data (see fig. 22 and/or fig. 24, paragraph [0133] and [0207]) captured by the eye tracking device (see fig. 24 and/or fig. 33, paragraphs [0208] and [0216]); and capture an image (see paragraph [0062] and [0128]) of a physical environment (see fig. 8, “forward, right, or rear view side of a vehicle”) by a first camera (see fig. 6, e.g., “front camera” and/or fig. 8) of the plurality of scene camera (see fig. 1 and/or fig. 6, e.g., “front camera, driver camera, other side camera”); wherein a field of view (see fig. 8, paragraph [0062]) of a first camera (see fig. 6 and/or fig. 8, e.g., “front camera”); and identify a region of interest in the physical environment (see fig. 8 and/or fig. 45, e.g., “rear-left view, rear-right view, front view”).
Yet, Mimar fails to explicitly wherein a field of view of a first camera overlaps the attention direction as claimed.
However, in the same field of endeavor, Tamaki teaches the well-known concept of wherein a field of view of a first camera overlaps the attention direction (see fig. 3 and/or fig. 6, paragraphs [0120] and [0123], where the base-line direction (e.g., “field of view”) of the compound-eye camera 21 coincides (e.g., “overlaps”) with the longitudinal direction of the face (e.g., “attention direction”)).
Therefore, before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains could recognize the advantage of modifying the proposed teachings of Mimar by further incorporating the proposed teachings of Tamaki above to perform such a modification to provide a driver gaze detection system and method that implements wherein a field of view of the camera overlaps the attention direction as well as to solve the problem in a case where the face orientation cannot be detected with sufficient accuracy without being affected by disturbance and so on as taught by Tamaki et al. (see Tamaki, paragraph [0012]), thus enhancing the accuracy of detecting the driver's face orientation.
Although Mimar discloses a non-transitory computer readable medium (see fig. 1 i.e. “hard disk”) storing instructions (see fig. 5 and/or fig. 9, e.g., “DRAM or flash memory”) executable by one or more processors (see fig. 6, e.g., “4 core CPU subsystem”), a method (see fig. 31), and a system (see fig. 6) comprising: a plurality of scene cameras (see fig. 1 and/or fig. 6, e.g., “front camera, driver camera, other side camera”); an eye tracking device (see fig. 24, fig. 33, paragraphs [0208] and [0216]); a display (see fig. 1 and/or fig. 6, e.g., “display”);one or more processors (see fig. 6, i.e. “4 core CPU subsystem”), and one or more non-transitory computer readable media storing instructions (see fig. 5, i.e. "DRAM Memory”) executable by the one or more processors (see fig. 5, i.e. “SoC Processor”) to: determine a camera pose of a first camera (see fig. 6, e.g., “front camera”) of the plurality of scene cameras (see fig. 1 and/or fig. 6, e.g., “front camera, driver camera, other side camera”); determine an attention direction of a user based on sensor data (see fig. 22 and/or fig. 24, paragraph [0133] and [0207]) captured by the eye tracking device (see fig. 24 and/or fig. 33, paragraphs [0208] and [0216]); and capture an image (see paragraph [0062] and [0128]) of a physical environment (see fig. 8, “forward, right, or rear view side of a vehicle”) by a first camera (see fig. 6, e.g., “front camera” and/or fig. 8) of the plurality of scene camera (see fig. 1 and/or fig. 6, e.g., “front camera, driver camera, other side camera”); wherein a field of view (see fig. 8, paragraph [0062]) of a first camera (see fig. 6 and/or fig. 8, e.g., “front camera”); and identify a region of interest in the physical environment (see fig. 8 and/or fig. 45, e.g., “rear-left view, rear-right view, front view”), the combination of teachings of Mimar and Tamaki fails to explicitly disclose identify a region of interest in the physical environment by projecting the attention direction into the image as claimed.
However, Tsukizawa further teaches the well-known concept of detect a region of interest (see fig. 3 unit 403 or unit 404) in the physical environment (see fig. 3, e.g., “subject’s face”) by projecting the attention direction (see paragraphs [0044] and [0068]) into the image (see fig. 3 unit 401).
Therefore, before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains could recognize the advantage of modifying the proposed combination of teachings of Mimar and Tamaki above by further incorporating the proposed teachings of Tsukizawa above to perform such a modification to provide visual axis direction detection device and visual line direction detection method that is a capable of identifying a region of interest in the physical environment by projecting the attention direction into the image as well as to solve the problem in a case where the shape and visual features of the inner and outer corners of the eye vary significantly between individuals. Therefore, to acquire an accurate gaze direction detection result, an apparatus needs to specify the relationships between the actual gaze direction and the inner and outer corners of the eye by measurement on a per subject basis, and correct various parameters for use in the calculation of the gaze direction. That is, the apparatus needs to be calibrated on a per subject basis (i.e. calibration), and, consequently, a problem arises that much processing is required and furthermore the apparatus is made complicated as taught by Tsukizawa (see Tsukizawa, paragraph [0007]), thus enhancing accuracy of a gaze direction result on a subject.
As per claim [2], most of the limitations have been noted in the above rejection of claim 1. In addition, Mimar discloses the non-transitory computer readable medium of claim 1 (see the above rejection of claim 1), further comprising instructions (see fig. 5 and/or fig. 9, e.g., “DRAM or flash memory”, paragraph [0147]) to: determine an object of interest in the image (see fig. 33).
As per claim [3], most of the limitations have been noted in the above rejection of claim 1. In addition, Mimar discloses the non-transitory computer readable medium of claim 2 (see the above rejection of claim 2), further comprising computer readable code (see paragraph [0147-0148]) to: integrate the object of interest (see fig. 33) into a digital map (see fig. 22 or fig. 36) of the physical environment (see fig. 45).
As per claim [4], most of the limitations have been noted in the above rejection of claim 1. In addition, Mimar discloses the non-transitory computer readable medium of claim 2 (see the above rejection of claim 2), further comprising instructions (see fig. 5 and/or fig. 9, e.g., “DRAM or flash memory”, paragraph [0147]) to: obtain data (see fig. 22) related to the object of interest (see fig. 33); and present the data (see fig. 22) on a display of the system (see fig. 1, e.g., “display”).
As per claim [5], most of the limitations have been noted in the above rejection of claim 1. In addition, Mimar discloses the non-transitory computer readable medium of claim 1 (see the above rejection of claim 1), further comprising instructions (see fig. 5 and/or fig. 9, e.g., “DRAM or flash memory”, paragraph [0147]) to obtain the image (see fig. 33) by selecting (see paragraph [0151]) from images (see fig. 45) captured by the plurality of scene cameras (see fig. 1 and/or fig. 6, e.g., “front camera, driver camera, other side camera”) based on the attention direction (see fig. 22 or fig. 49).
As per claim [6], most of the limitations have been noted in the above rejection of claim 1. In addition, Mimar discloses the non-transitory computer readable medium of claim 1 (see the above rejection of claim 1), further comprising instructions (see fig. 5 and/or fig. 9, e.g., “DRAM or flash memory”, paragraph [0147]) to: determine a gaze direction (see fig. 22) based on the sensor data (see fig. 22 or fig. 38), wherein the attention direction (see figs. 39-40) is determined based on the gaze direction (see fig. 22 or fig. 49).
As per claim [7], most of the limitations have been noted in the above rejection of claim 1. In addition, Mimar discloses the non-transitory computer readable medium of claim 6 (see the above rejection of claim 6), wherein the attention direction is determined based on a target screen location of the gaze direction (see fig. 22).
As per claim [9], the method of claim 8, is analogous to claim 2, which is performed by claim 9.
As per claim [10], the method of claim 9, is analogous to claim 3, which is performed by claim 10.
As per claim [11], the method of claim 9, is analogous to claim 4, which is performed by claim 11.
As per claim [12], the method of claim 8, is analogous to claim 5, which is performed by claim 12.
As per claim [13], the method of claim 8, is analogous to claim 6, which is performed by claim 13.
As per claim [14], the system of claim 13, is analogous to claim 7, which is performed by claim 14.
As per claim [16], the system of claim 15, is analogous to claim 2, which is performed by claim 16.
As per claim [17], the system of claim 16, is analogous to claim 3, which is performed by claim 17.
As per claim [18], the system of claim 16, is analogous to claim 4, which is performed by claim 18.
As per claim [19], the system of claim 16, is analogous to claim 5, which is performed by claim 19.
As per claim [20], the system of claim 15, is analogous to claim 6, which is performed by claim 20.
Conclusion
5. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Akay (US Pub. No.: 2015/0221341 A1) discloses a system and method for enhance time-lapse video generation using panoramic imagery.
Di Censo et al. (US Pub. No.: 2015/0006278 A1) discloses an apparatus and method for detecting a driver’s interest in an advertisement by tracking driver eye gaze.
Hong (US Pub. No.: 2008/0181591 A1) disclose a camera posture estimation device, vehicle, and camera posture estimation method.
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Richard Carter whose telephone number is (571)270-1220. The examiner can normally be reached M-F 8:30 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached on 571-272-2988. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.B.C/Examiner, Art Unit 2485
/JAYANTI K PATEL/Supervisory Patent Examiner, Art Unit 2485
July 2, 2026
Prosecution Insights