Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 101
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the claim limitation is directed towards ineligible patent subject matter.
Claim 7, states:
7. A computer-readable storage medium containing instructions that, when executed by a computer, cause said computer to perform operations, comprising: […]
The specification doesn’t limit what “A computer-readable storage medium” is.
See MPEP 2106.03:
Non-limiting examples of claims that are not directed to any of the statutory categories include:
Transitory forms of signal transmission (often referred to as “signals per se”), such as a propagating electrical or electromagnetic signal or carrier wave; and
Therefore the claim is not directed towards patent eligible subject matter.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 - 8 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "wherein the image data contain an image of an occupant of the vehicle from slightly different perspectives;" in lines 5-6. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "wherein the image data contain an image of an occupant of the vehicle from slightly different perspectives;" in lines 4-5. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites the limitation "wherein the image data contain an image of an occupant of the vehicle from slightly different perspectives;" in lines 6-7. There is insufficient antecedent basis for this limitation in the claim.
The examiner notes that claims 2-6 are considered indefinite for depending upon claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-8 are rejected under 35 U.S.C. 103 as being unpatentable over Wantland et al. (US 20210042950 A1) as cited in an IDS in view of Arora et al. (US 20210397859 A1) as cited in an 892.
Regarding claim 1, Wantland teaches a computer-implemented method for image processing in video conferences […](See abstract. ¶40, “(i) segmentation masks, and (ii) depth maps, to provide depth-aware editing and/or real-time depth-aware processing of specific objects or features in a photo or video.” ¶41, “The depth-aware processes described herein could additionally or alternatively be implemented by a video conference application, and/or other types of applications.” ), wherein the […] has a first camera and a second camera (¶31, “ For example, in some embodiments, a method of depth estimation from pairs of stereo images includes capturing, at a pair of cameras, a first image and a second image of a scene. ”¶27, “stereo imaging may be utilized to generate a depth map. In such embodiments, a depth map may be obtained by correlating left and right stereoscopic images to match pixels between the stereoscopic images.”), the method comprising:
in a reception step, receiving first image data from the first camera and second image data from the second camera, wherein the image data contain an image of an […]from slightly different perspectives (¶27, “[…] For example, a disparity between the location of the pixel in the left image and the location of the corresponding pixel in the right image may be used to calculate the depth information using binocular disparity techniques. An image may be produced that contains depth information for a scene, such as information related to how deep or how far away objects in the scene are in relation to a camera's viewpoint. […]” ¶27 notes that the cameras are capturing from different perspectives, in this context they’re considered slightly different for the purpose of producing depth. ¶44, “The computing device determines depth information (e.g., a depth map) for the scene, as shown by block 104. The depth information for the scene can be determined based at least in part on the first image.”);
in a first extraction step, extracting a depth information map of the first image data using at least one of the two image data (¶27, “[…] For example, a disparity between the location of the pixel in the left image and the location of the corresponding pixel in the right image may be used to calculate the depth information using binocular disparity techniques. An image may be produced that contains depth information for a scene, such as information related to how deep or how far away objects in the scene are in relation to a camera's viewpoint. […]” ¶44, “The computing device determines depth information (e.g., a depth map) for the scene, as shown by block 104. The depth information for the scene can be determined based at least in part on the first image.”);
in a second extraction step, extracting features in the first image data;
in a classification step, classifying the features as classified features, wherein the classified features include at least the occupant (¶20, “For instance, a CNN may be trained and subsequently utilized to solve a semantic segmentation task. The specific segmentation task may be to a binary or multi-level prediction mask that separates objects in the foreground of an image from a background area or areas in an image. Prediction masks can correspond to estimated segmentations of an image (or other estimated outputs) produced by a CNN.” ¶44, “The computing device also determines segmentation data for the first image, as shown by block 106. Then, based at least in part on (a) the depth information, and (b) the segmentation data, the computing device processes the first image to generate an edited version of the first image, as shown by block 108.”);
in a fusion step, fusing the depth information map and the classified features to form a three-dimensional model;
in a synthesis step, synthesizing the first image data and the three-dimensional model to form a synthesized image (¶40,“[…] a combination of: (i) segmentation masks, and (ii) depth maps, to provide depth-aware editing and/or real-time depth-aware processing of specific objects or features in a photo or video.” ¶44, “Then, based at least in part on (a) the depth information, and (b) the segmentation data, the computing device processes the first image to generate an edited version of the first image, as shown by block 108.” ¶44-¶45, “Examples of depth-aware image processing that may be implemented at block 108 include selective object removal, selective blurring, the addition of three-dimensional (3D) AR graphic objects and animations, object-specific zoom, generation of interactive image content with parallax visualization (e.g., a “pano-selfie”), bokeh effects in still images, video and real-time “live-view” interfaces, focal length adjustment in post-processing of still images and video, software-based real-time simulation of different focal lengths in a “live-view” interface, and/or the addition of virtual light sources in a real-time “live-view” interface and/or in image post-processing, among other possibilities.” Paragraph 45, is describing a fusion step of fusing a 3d model with an image from depth ware image processing to form a synthesize image); and
in a display step, displaying the synthesized image on a display device (See ¶85. Fig. 5A-5C: There is displayed a synthesized image. ) but doesn’t explicitly disclose:
in a vehicle […] the vehicle has a first camera and a second camera; […]occupant of the vehicle […].
Arora teaches in a vehicle […] the vehicle has a first camera and a second camera; […]occupant of the vehicle […] (See Fig. 1, elements 102A and 102B both shows cameras on an occupant from different angles. ¶12 for further details).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Wantland in view of Arora as “Using this determined distance between cameras 102 and the occupant's eyes as well as the location of the occupant's eyes in the captured images, vehicle computing system 104 may place the occupant's eyes in 3D space relative to cameras 102. That is, vehicle computing system 104 and/or the camera system may determine a location of the occupant's eyes within the interior of vehicle 100 relative to one or more of cameras 102” (¶17 Arora).
Regarding claim 2, Wantland in view of Arora the computer-implemented method as claimed in claim 1, wherein at least one of the two cameras has the ability to generate image data from the infrared spectral range (Arora ¶13, “In some examples, cameras 102 may be one or more infrared cameras with a high field-of-view and shallow depth of focus, and may be a backlit infrared camera oriented to point generally towards one or more seats of vehicle 100”).
Regarding claim 3, Wantland in view of Arora the computer-implemented method as claimed in claim 1, wherein the synthesis step comprises cropping the image of the occupant and/or replacing a background (See Wantland ¶65, “On the other hand, to selectively zoom out on an object after image capture without affecting the apparent depth of the object's background, an editing application will typically need to generate replacement background image content to replace portions of the image that are uncovered when the size of the selected object is reduced. Depth information could be used to generate the replacement image content, as described above.”).
Regarding claim 4, Wantland in view of Arora teaches the computer-implemented method as claimed in claim 1, wherein the synthesis step comprises modifying illumination properties and/or reflectivity properties of the classified features of the three-dimensional model if at least one previously defined constraint is satisfied (See Wantland ¶74-77, the image could be too bright or too dark […] change the lighting.).
Regarding claim 5, Wantland in view of Arora teaches the computer-implemented method as claimed in claim 4, wherein the previously defined constraint corresponds to overexposure of a classified feature or a rapid change in the exposure of a classified feature (See Wantland ¶74-77, the image could be too bright or too dark […] change the lighting.).
Regarding claim 6, Wantland in view of Arora teaches the computer-implemented method as claimed in claim 4, wherein the modification of the illumination properties and/or reflectivity properties of the classified features comprises: in a third extraction step, extracting a reflectivity map on the basis of the three-dimensional model; in a fourth extraction step, extracting the illumination properties and reflectivity properties of the classified features on the basis of the reflectivity map; in a modification step, modifying the illumination properties and reflectivity properties of the classified features to form modified illumination properties and modified reflectivity properties; and in a generation step, generating the synthesized image such that the illumination properties and reflectivity properties are replaced by the modified illumination properties and modified reflectivity properties (¶77, “In a further aspect, applying the depth-variable light-source effect could involve the computing device using a depth map for the image to determine depth information for at least one background area in the image (e.g., as identified by a segmentation mask for the image). Then, based at least in part on (a) the depth information for the at least one background area, (b) the coordinates of the light source, and (c) coordinates of the at least one object in the three-dimensional image coordinate frame, the computing device can generate shadow data for the background area corresponding to the at least one object and the light source. The shadow data may be used to modify the image with shadows from objects that correspond to the virtual light source in a realistic manner.”).
Claim 7 recites similar limitations to that of claim 1 and thus is rejected under similar rationale as detailed above.
Claim 8 recite similar limitations to that of claim 1 and thus is rejected under similar rationale as detailed above but claim 1 doesn’t explicitly disclose: at least one non-volatile, computer-readable storage medium that is communicatively connected to the at least one processor, wherein the storage medium stores instructions in a programming language for performing a computer-implemented operation; a first camera that is communicatively connected to the at least one processor; a second camera that is communicatively connected to the at least one processor; a communication means that is communicatively connected to the at least one processor; and a display device that is communicatively connected to the at least one processor.
Wantland teaches at least one non-volatile, computer-readable storage medium (See Fig. 9 element 904. ¶106, “Data storage 904 can include one or more non-transitory computer-readable storage media that can be read and/or accessed by at least one of one or more processors 903. The one or more computer-readable storage media can include volatile and/or non-volatile storage components, such as optical, magnetic, organic or other memory or disc storage, which can be integrated in whole or in part with at least one of one or more processors 903. In some examples, data storage 904 can be implemented using a single physical device (e.g., one optical, magnetic, organic or other memory or disc storage unit), while in other examples, data storage 904 can be implemented using two or more physical devices.” The previously cited and bolded memories are non-volatile computer readable storage mediums.) that is communicatively connected to the at least one processor (See Fig. 9 one or more processor element 903 and data storage 904 are communicatively connected via a bus.¶102: processor and data storage all communicatively connected via a bus or a network or other connection mechanism.), wherein the storage medium stores instructions in a programming language for performing a computer-implemented operation (¶6, In another aspect, an example computer readable medium comprises program instructions that are executable by a processor to perform functions comprising […]“ The instructions are in a programming language.); a first camera that is communicatively connected to the at least one processor; a second camera that is communicatively connected to the at least one processor (¶102, “Computing device 900 may include a user interface module 901, a network communications module 902, one or more processors 903, data storage 904, one or more cameras 918, one or more sensors 920, and power system 922, all of which may be linked together via a system bus, network, or other connection mechanism 905.” Also see Fig. 9); a communication means that is communicatively connected to the at least one processor; and a display device that is communicatively connected to the at least one processor (See ¶109, “Computing device 900 may include a user interface module 901, a network communications module 902, one or more processors 903, data storage 904, one or more cameras 918, one or more sensors 920, and power system 922, all of which may be linked together via a system bus, network, or other connection mechanism 905.” Fig. 9. ¶103, “User interface module 901 can be operable to send data to and/or receive data from external user input/output devices. For example, user interface module 901 can be configured to send and/or receive data to and/or from user input devices such as a touch screen, […] User interface module 901 can also be configured to provide output to user display devices, such as one or more cathode ray tubes (CRT), liquid crystal displays, light emitting diodes (LEDs), displays using digital light processing (DLP) technology […]” Said another way, the user interface module provides a connection from the display to the other components as well by being connected to the user interface module.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Rober et al. (US Patent No. 10,482,669) as cited in an 892:
Col. 13 line 10-14, “The voice input may, for example, be used for voice control
of the VR system, or for communicating with other passengers wearing HMDs, or for external communications such as phone calls and teleconferencing through the VR controller 710 or vehicle 700 systems.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT J CRADDOCK whose telephone number is (571)270-7502. The examiner can normally be reached Monday - Friday 10:00 AM - 6 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Devona E Faulk can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT J CRADDOCK/Primary Examiner, Art Unit 2618