Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
2. This is the initial Office Action based on the application filed on July 25, 2025. The Examiner acknowledges the following:
3. Claims 1 – 15 were filed by Applicant.
4. The drawings filed on the same date are accepted by the Examiner.
5. Current claims 1 – 15 are pending and they are being considered for examination.
Information Disclosure Statement
6. The IDS document filed on filed on 07/25/2025 is acknowledged by the Examiner.
Priority
7. Priority data is based on a PCT/JP patent application PCT/KR2023/020313 with date of 12/11/2023, which refers to a previous Korean patent applicationKR-10-2023-001236, with priority date of 01/26/2023. Certified copies were filed to the office on 08/17/2025.
Objection to Drawings
8. The drawing/photograph disclosure is objected to because: Fig 3 includes a typographic error. The element on the very right includes the error. It says “obtain”, instead of “obtain”. Corrected drawings in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Appropriate correction is expected.
Claim Rejections - 35 USC § 101
9. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Regarding claim 15:
1. Claim 15 is rejected under 35 U. S. C. 101 because the claimed invention is directed to non-statutory subject matter.
2. Claim 15 recites,
“A computer-readable recording medium having recorded thereon at least one program which, when executed by at least one processor of an electronic device causes…”
3. An Examiner is obliged to give claims their broadest reasonable interpretation consistent with the specification during examination. The broadest reasonable interpretation of a claim drawn to a computer readable medium (also called machine readable medium and other such variations) typically covers forms of non-transitory tangible media and transitory propagating signals per se in view of the ordinary and customary meaning of computer readable media, particularly when the specification is silent. See MPEP 2111.01. When the broadest reasonable interpretation of a claim covers a signal, per se, the claim must be rejected under 35 U.S.C. § 101 as covering non-statutory subject matter.
4. However, the Examiner respectfully submits a claim drawn to such a computer-readable recording medium that covers both transitory and non-transitory embodiments may be amended to narrow the claim to cover only statutory embodiments to avoid a rejection under 35 U.S.C. § 101 by adding the limitation “non-transitory” to the claim. Such an amendment would typically not raise the issue of new matter, even when the specification is silent because the broadest reasonable interpretation relies on the ordinary and customary meaning that includes signals per se. For additional information, please see the Patents’ Official Gazette notice published February 23, 2010 (1351 OG 212).
Appropriate correction is needed
Claim Rejections - 35 USC § 102
10. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 – 3, 7 – 11 and 13 – 15 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by “Oskar Linde et al., US 2019/0110039A1, hereinafter Linde”.
Regarding claims 1, 11 and 15:
Line teaches a head-mounted display (HMD) comprising: an imaging assembly that includes a plurality of cameras positioned at different locations on the HMD and oriented to capture images of different portions of a local area surrounding the HMD, the plurality of cameras each having overlapping fields of view with adjacent cameras, and the imaging assembly is configured to: generate exposure settings for each of the plurality of cameras, and capture images of the local area in accordance with imaging instructions; a controller configured to: generate imaging instructions using the exposure settings, the imaging instructions causing respective midpoints of exposure times for each of the plurality of cameras to occur at a same time value for each of the captured images, provide the imaging instructions to the imaging assembly, and determine a location of the HMD in the local area using the captured images, wherein the imaging assembly comprises four cameras, each camera having a field of view within a range of 120 to 180 degrees, wherein a first camera and a second camera have a substantially parallel orientation such that the fields of view of the first camera and the second camera provide stereoscopic imaging of a portion of the local area and, wherein a third camera and a fourth camera have a different orientation such that only a portion of the fields of view of the third camera and the fourth camera overlap.
Regarding claim 1:
As for claim 1, Linde teaches,
An electronic device (Fig 1, HMD (head-mounted display) 100. See [0015]) comprising:
a first camera configured to capture a base image with respect to a space corresponding to the electronic device (Fig 1, camera 115. See [0015;0019; 0020]);
a second camera configured to capture a line-of-sight image corresponding to a line-of-sight direction; a motion sensor (Fig 1, camera 120. See [0015; 0019; 0020]);
at least one processor (HMD controller generates imaging instruction for the plurality of cameras by centering the exposure length of each camera about the center time point (See [0026]). It is configured to determine the depth information for the one or more objects in the local area based in one or more captured images from the imaging assembly (See [0027; 0028]). Fig 3, HMD controller 350. See [0047]); and a memory (Fig 3, application store 355 stores one or more applications for execution by the HMD controller 350. See [0048]) storing at least one instruction which, when executed by the at least one processor (Linde, Claims 1 and 20 includes one or more processors to perform the operation of the HMD), causes the electronic device to: obtain light quantity information about a light quantity associated with the space, based on the base image (Fig 1, cameras 115, 120, 125, 130 detect the amount of light incident on the image sensor and based on the detected light, the camera controller determines appropriate exposure settings based on the amount of light detected from a first image or a first set of frames to map the object within the environmental surrounding the HMD 305. See [0018; 0022; 0052]); and
determine, based on motion information (Fig 1, position sensor 140 detects motion of the HMD 100. They include one or more accelerometers, one or more gyroscopes, one or more magnetometers or any suitable type of sensor that detects motion. See [0016]) obtained by the motion sensor and the light quantity information (Fig 1, the image assembly comprises cameras positioned to capture a portion of the local area, wherein image information may include one or more images, audio information, video information metadata and the amount of light detected by the cameras. See [0018; 0022]), an exposure value for capturing the line-of-sight image (The camera controller of each camera 115, 120, 125, 130 determines exposure settings for the camera. The exposure settings of a camera determines how light captured by the camera is collected by the sensor. Each camera 115, 120, 125, 130 detects the amount of light incident on the sensor, and based on the detected light, the camera controller determines appropriate exposure settings. The exposure settings may include, e.g., aperture size, shutter speed, gain, or some combination thereof. The aperture size controls the amount of light that reaches the sensor. The shutter speed is the length of time that the sensor is exposed to light (i.e., an exposure length). The gain is the sensitivity of the sensor to the light. In the embodiment of FIG. 1, the aperture is a fixed size while the gain and exposure are variable. Using the amount of detected light, the camera controller for each camera determines the aperture size, shutter speed, and gain settings for each camera. Furthermore, the camera controller may associate a first exposure value with the high priority area and a different exposure value with the low priority area. An exposure value represents a combination of a camera's shutter speed and f-number (i.e., the ratio of focal length to aperture size), such that all combinations that yield the same exposure have the same exposure value. The camera controller determines exposure settings for the camera for the high priority area such that the exposure settings expose for the first exposure value. One or more of the cameras 115, 120, 125, 130 may have exposure settings based on exposure values (See [0024]). As for line-of-sight. See [0049]).
Regarding Claim 11:
The rejection of claim 1 is incorporated herein. Claim 11 pertains to the method steps performed as to operate the electronic device as disclosed in claim 1. In order to operate an electronic device as disclosed in claim 1, it would have necessitated to do the method of claim 11. Claim 11 includes similar limitations as disclosed in claim 1. As for claim 11, Linde teaches,
the method (Fig 6 shows the method for determining the location of the HMD 100 within a local area. See [0067 – 0073]) comprising:
obtaining light quantity information about a light quantity associated with a space corresponding to the electronic device, based on a base image with respect to the space (Fig 1, cameras 115, 120, 125, 130 detect the amount of light incident on the image sensor and based on the detected light, the camera controller determines appropriate exposure settings based on the amount of light detected from a first image or a first set of frames to map the object within the environmental surrounding the HMD 305. See [0018; 0022; 0052]); and
based on the light quantity information and motion information (Fig 1, position sensor 140 detects motion of the HMD 100. They include one or more accelerometers, one or more gyroscopes, one or more magnetometers or any suitable type of sensor that detects motion. See [0016]) obtained by a motion sensor included in the electronic device (Fig 1, the image assembly comprises cameras positioned to capture a portion of the local area, wherein image information may include one or more images, audio information, video information metadata and the amount of light detected by the cameras. See [0018; 0022]), determining an exposure value for capturing a line-of-sight image corresponding to a line-of-sight direction The camera controller of each camera 115, 120, 125, 130 determines exposure settings for the camera. The exposure settings of a camera determine how light captured by the camera is collected by the sensor. Each camera 115, 120, 125, 130 detects the amount of light incident on the sensor, and based on the detected light, the camera controller determines appropriate exposure settings. The exposure settings may include, e.g., aperture size, shutter speed, gain, or some combination thereof. The aperture size controls the amount of light that reaches the sensor. The shutter speed is the length of time that the sensor is exposed to light (i.e., an exposure length). The gain is the sensitivity of the sensor to the light. In the embodiment of FIG. 1, the aperture is a fixed size while the gain and exposure are variable. Using the amount of detected light, the camera controller for each camera determines the aperture size, shutter speed, and gain settings for each camera. Furthermore, the camera controller may associate a first exposure value with the high priority area and a different exposure value with the low priority area. An exposure value represents a combination of a camera's shutter speed and f-number (i.e., the ratio of focal length to aperture size), such that all combinations that yield the same exposure have the same exposure value. The camera controller determines exposure settings for the camera for the high priority area such that the exposure settings expose for the first exposure value. One or more of the cameras 115, 120, 125, 130 may have exposure settings based on exposure values (See [0024]). As for line-of-sight. See [0049]).
Regarding claim 15:
The rejection of claims 1 and 11 is incorporated herein. Claim 15 pertains to a computer-readable recording medium having recorded thereon at least one program which, when executed by the at least one processor of an electronic device, cause the electronic device to perform the method of claim 11 as to operate the device of claim 1. The limitations of claim 15 were already discussed in the rejection of claims 1 and 11 above. As for the “computer-readable recording medium” see Linde claim 20 and paragraphs ([0076 – 0078]).
Regarding claims 2 and 3:
The rejection of claim 1 is incorporated herein. As for claims 2 and 3 limitations, Linde Fig 1 shows 4 cameras 115, 120, 125 and 130, which have a field-of-view within the range of 120 to 180 degrees in which, a field of view of 50 to 120 degrees corresponds to a wide field of view and a field of view larger that 120 degrees is referred as a fish eye field of view. Linde teaches that the cameras have different fields of view (See [0020] and Linde claim 2).
Regarding Claims 7 and 13:
The rejection of claims 1 and 11 is incorporated herein. Linde teaches both claims. Linde teaches a prediction based on the motion information based on the future position of the HMD in a local area by using information of the IMU and a model and determining a camera exposure setting for a plurality of cameras 115, 120, 125 and 130 on the basis of a predicted future position as indicated in Linde claims 17 and 18.
Regarding Claim 8:
The rejection of claims 1 and 7 is incorporated herein. As for claim 8 limitations, Linde teaches in Fig 3, engine 365 also updates a local area model using the information from the imaging assembly 315 and the tracking module 360. The local area model may represent a mapping function of the area surrounding some or all of the HMD 305 (i.e., the “local area”) based on the location of the HMD 305 within the local area. The local area model provides the exposure settings for each camera of the imaging assembly 315 for different positions and orientations of the HMD 305 and allows the exposure settings to be adjusted as the location and orientation of the HMD 305 changes within the environment (See [0052]). In other words, Linde teaches an imaging assembly which captures images in various ranges of light within a local area around the HMD 100 and determines the appropriate exposure settings for each of the cameras of the imaging assembly on the basis of the new position and orientation of the HMD 100.
Fig 5 shows a method for synchronizing image capture of the cameras 115, 120, 125, 130 on the HMD 100, according to one embodiment. As described with regards to FIG. 1, the cameras 115, 120, 125, 130 are configured to have individual exposure settings rather than a global exposure setting applied to the imaging assembly. This configuration enables a first camera to have a much longer exposure than a second camera, allowing the imaging assembly to capture a varying range of light within a local area surrounding the HMD 100. The HMD 100 selects different parts of the local area to drive exposure towards to gather desired information from the local area. As illustrated in FIG. 5, each camera 115, 120, 125, 130 has a different length of exposure frame 505, 510, 515, 520, respectively (See [0062]). Additionally, it teaches a center time point determined by the HMD controller synchronizes the exposure settings of each camera 115, 120, 125, 130 by centering the exposure length of each camera 115, 120, 125, 130 about the center time point. In other words, a midpoint of each exposure length is aligned at the same time point. This configuration accommodates the varying exposure lengths of the cameras 115, 120, 125, 130 and ensures that the same frame is captured by each camera (See [0026; 0062]).
Regarding Claim 9:
The rejection of claim 1 is incorporated herein. Linde teaches claim 1. As for claim 9 limitations, Linde teaches an imaging assembly which generates image information by using the image information and/or audio information captured from the local area around the HMD 100 (See [0018]) or some combination thereof. Image information may include depth information of one or more objects in the local area and/or amount of light detected by the cameras 115, 120, 125, 130. Additionally, the HMD controller is configured to update a local area model for the HMD 100. The local area model includes depth information, exposure settings, or some combination thereof of the environment surrounding the HMD 100. In the embodiment of FIG. 1, the local area model represents a mapping function of depth information and exposure settings of the environment of the HMD 100 based on the location of the HMD 100 within the environment (See [0028]).
Regarding Claims 10 and 14:
The rejection of claims 1, 11 and 13 is incorporated herein. Linde teaches that each camera controller operates independently from one another, such that each of the controllers is capable of determining different exposure settings for the corresponding cameras 115, 120, 125 and 130 with respect to other cameras. For example, the shutter speed for each camera may vary to expose the sensor to more or less light. This configuration enables a first camera to have a much longer exposure than a second camera, which allows the imaging assembly to capture a varying range of light within a local area and to expose different parts of the local area to gather desired information from the local area . As an example, a local area of a room may have a first side of the room that is well-lit (e.g., a window that allows a large amount of light into the room) while a second side of the room is shrouded in shadows. An imaging assembly having a global exposure setting may determine exposure settings that are appropriate for the first side or the second side of the room but may cause the imaging assembly to over-expose or under-expose the other side of the room, thereby preventing the imaging assembly from capturing images having the desired information of the entire local area (See [0023]).
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103, which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 5, 6 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over “Oskar Linde et al., US 2019/0110039A1, hereinafter Linde” in view of “Yevhenii Yakishyn et al., US 2022/0159174 A1, hereinafter Yakishyn”. (Note: Yakishyn art is from the IDS document).
Regarding Claims 4 and 12:
The rejection of claims 1 and 11 is incorporated herein. As for claims 4 and 12 limitations, Linde teaches claims 1 and 11 limitations, but it is silent about images with the feature point position information, which in the same field of endeavor is taught by Yakishyn. Yakishyn a method for controlling an electronic device by recognizing movement of an object., dividing the obtained one image into middle and peripheral zones and extracting one or more feature points of the object that are in the peripheral/edge of the image and recognizing movement of the object based on the extracted one or more features and control the electronic device based on the recognized movement (See [0111 and Yakishyn claim 1).
By modifying Linde to include feature recognition of an object imaged by the electronic device as taught by Yakishyn, that would allow the use to trach the movement of the object based on the feature point position when object is located on the periphery of the image.
Regarding Claims 5 and 6:
The rejection of claim 1 is incorporated herein. Linde teaches claim 1 and additionally that HMD 100 (Fig 1), wherein the position sensor 330 (Fig 3) includes one or more accelerometers, one or more gyroscopes, one or more magnetometers and other suitable type of sensor that detect motion (See [0016; 0042; 0043]). Using the local area model and position information from IMU 135 such as velocity vector, it can predict a future location and orientation of the HMD 100 (See [0028; 0034]). Linde teaches most of the limitations of claim 5, but it is silent about “image sequence sequentially obtained..”, which in the same field of endeavor is taught by Yakishyn, wherein the images are capture as a video or an image sequence (See [0049]). It teaches in Fig 2 that the processor 1300 may execute one or more instructions configuring the feature point extraction module 1540 from among the programs stored in the storage 1500, to extract one or more feature points (or key points or key features) of the object included in the peripheral zone or the edge image (See [0111]).
Therefore, by modifying Linde to include feature recognition of an object imaged by the electronic device as taught by Yakishyn, that would allow the use to track the movement of the object based on the feature point position as a first position (first time point) and for predicting a future position of the HMD 100 when using a local area model and then, determining the appropriate exposure for any future predicted position of the HMD 100 (See [0073]).
Conclusion
12. The prior art is made of record and not relied upon is considered pertinent to applicant’s disclosure.
1. O. Linde et al., US 2019/0110039 A1 – it teaches a head-mounted display (HMD) comprising: an imaging assembly that includes a plurality of cameras positioned at different locations on the HMD and oriented to capture images of different portions of a local area surrounding the HMD, the plurality of cameras each having overlapping fields of view with adjacent cameras, and the imaging assembly is configured to: generate exposure settings for each of the plurality of cameras, and capture images of the local area in accordance with imaging instructions; a controller configured to: generate imaging instructions using the exposure settings, the imaging instructions causing respective midpoints of exposure times for each of the plurality of cameras to occur at a same time value for each of the captured images, provide the imaging instructions to the imaging assembly, and determine a location of the HMD in the local area using the captured images, wherein the imaging assembly comprises four cameras, each camera having a field of view within a range of 120 to 180 degrees, wherein a first camera and a second camera have a substantially parallel orientation such that the fields of view of the first camera and the second camera provide stereoscopic imaging of a portion of the local area and, wherein a third camera and a fourth camera have a different orientation such that only a portion of the fields of view of the third camera and the fourth camera overlap.
Contact
13. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARLY CAMARGO whose telephone number is (571)270-3729. The examiner can normally be reached on 6:00AM - 10PM, M-F, EST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Lin Ye can be reached on (571)272-7372. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARLY S CAMARGO/Primary Examiner, Art Unit 2638
