DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claim(s) 1-3, 6-8, and 12-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Park et al (US 2018/0081171) in view of Park et al (US 2018/0005421).
As to claim 1, Park171 teaches a control apparatus (processor 120, fig. 1) configured to control a display apparatus (display 160, fig. 1) for displaying a partial image based on a partial area of an omnidirectional image or hemispherical image on a display surface ([0114] FIGS. 6A and 6B, the electronic device 101 may display a first part 621 of a wide-angle image, that is, an entire image 620), the control apparatus comprising: a memory storing instructions ([0219] a storage medium storing instructions); and a processor configured to execute the instructions to ([0219] the instructions are configured to enable at least one processor to perform at… if the instructions are executed by the at least one processor): generate the partial image by performing transformation processing for the partial area of the omnidirectional image or hemispherical image ([0219] displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part, [0114] FIGS. 6A and 6B, the electronic device 101 may display a first part 621 of a wide-angle image, that is, an entire image 620), detect a first position of a line of sight of a user on the display surface, generate a first partial image based on the first position and display the first partial image on the display surface ([0219] determining a part corresponding to a user gaze on a first frame in the first frame of the video; determining a decoding-target part, which includes the part corresponding to the user gaze on the first frame…decoding the decoding-target part; and displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part, see entire paragraph of [0115], figs. 6A and 6B), switch the partial image (fig. 6A illustrates switching the partial image from 622 to 621), wherein the partial image is an image generated by performing the transformation processing for the partial area of the omnidirectional image or hemispherical image ([0219] determining a part corresponding to a user gaze on a first frame in the first frame of the video; determining a decoding-target part, which includes the part corresponding to the user gaze on the first frame…decoding the decoding-target part; and displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part) of one viewpoint (for example second part 622, fig. 6A) among images captured at a plurality of viewpoints (plurality of viewpoints including the first part 621 and the second part 622, fig. 6A)
Park171 does not teach generating … and displaying the first partial image on the display surface in a case where the first position is located within a predetermined area for a predetermined duration.
However, Park421 teaches generating a first partial image based on the first position and displaying the first partial image on the display surface in a case where the first position is located within a predetermined area for a predetermined duration ([0332] When the user gazes at the second virtual image 2450 for a predetermined time, the controller 480 may display a detailed virtual image 2451 including detailed information of the first virtual image 2450, fig. 24A).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Park171 to teach, generating a first partial image based on the first position and displaying the first partial image on the display surface in a case where the first position is located within a predetermined area for a predetermined duration, as suggested by Park421. The motivation would have been in order to prevent the user from “feeling dizzy or motion sick.”
As to claim 2, Park171 in view of Park421 teaches the control apparatus, wherein the processor is configured to generate the first partial image with the first position as a center (Park171: [0219] determining a part corresponding to a user gaze on a first frame in the first frame of the video; determining a decoding-target part, which includes the part corresponding to the user gaze on the first frame…decoding the decoding-target part; and displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part, see entire paragraph of [0115], figs. 6A and 6B).
As to claim 3, Park171 in view of Park421 teaches the control apparatus, wherein the processor is configured to generate a second partial image with a second position as a center that is moved from an original position toward the first position by a predetermined amount every predetermined time, and to display the second partial image on the display surface (Park171: [0115] While viewing the first part 621, the user may turn his/her head in the left direction. Accordingly, the electronic device 101 may sense a rotation 612 in the left direction. The electronic device 101 may display a left eye image 641 and a right eye image 642 corresponding to a second part 622).
As to claim 6, Park171 in view of Park421 teaches the control apparatus, wherein the transformation processing is perspective projection transformation (Park171: determining a decoding-target part, which includes the part corresponding to the user gaze on the first frame and has a size greater than or equal to the part corresponding to the user gaze on the first frame; decoding the decoding-target part; and displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part.).
As to claim 7, Park171 in view of Park421 teaches the control apparatus, wherein coordinates at the first position can be converted into coordinates in the omnidirectional image or hemispherical image (Park171: determining a part corresponding to a user gaze on a first frame in the first frame of the video; determining a decoding-target part, which includes the part corresponding to the user gaze on the first frame and has a size greater than or equal to the part corresponding to the user gaze on the first frame; decoding the decoding-target part).
As to claim 8, Park171 in view of Park421 teaches the control apparatus, wherein images captured at the plurality of viewpoints include a right-eye image and a left-eye image (Park171: [0219] displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part).
As to claim 12, Park171 in view of Park421 teaches a display apparatus comprising: the control apparatus according to claim 1 (Park171: processor 120, fig. 1); and a display unit (Park171: display 160).
As to claim 13, Park171 teaches a control method (processor 120, fig. 1) configured to control a display apparatus (display 160, fig. 1) for displaying a partial image based on a partial area of an omnidirectional or hemispherical image on a display surface ([0114] FIGS. 6A and 6B, the electronic device 101 may display a first part 621 of a wide-angle image, that is, an entire image 620), the control method comprising the steps of:
generating the partial image by performing transformation processing for the partial area of the omnidirectional or hemispherical image ([0219] displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part, [0114] FIGS. 6A and 6B, the electronic device 101 may display a first part 621 of a wide-angle image, that is, an entire image 620),
detecting a first position of a line of sight of a user on the display surface, and generating a first partial image based on the first position and display the first partial image on the display surface ([0219] determining a part corresponding to a user gaze on a first frame in the first frame of the video; determining a decoding-target part, which includes the part corresponding to the user gaze on the first frame…decoding the decoding-target part; and displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part, see entire paragraph of [0115], figs. 6A and 6B), and
switching the partial image (fig. 6A illustrates switching the partial image from 622 to 621), wherein the partial image is an image generated by performing the transformation processing for the partial area of the omnidirectional image or hemispherical image ([0219] determining a part corresponding to a user gaze on a first frame in the first frame of the video; determining a decoding-target part, which includes the part corresponding to the user gaze on the first frame…decoding the decoding-target part; and displaying a left eye image and a right eye image of the part corresponding to the user gaze on the first frame among the decoding-target part) of one viewpoint (for example second part 622, fig. 6A) of one viewpoint (or example second part 622, fig. 6A) among images captured at a plurality of viewpoints (plurality of viewpoints including the first part 621 and the second part 622, fig. 6A).
Park171 does not teach generating … and displaying the first partial image on the display surface in a case where the first position is located within a predetermined area for a predetermined duration.
However, Park421 teaches generating a first partial image based on the first position and display the first partial image on the display surface in a case where the first position is located within a predetermined area for a predetermined duration ([0332] When the user gazes at the second virtual image 2450 for a predetermined time, the controller 480 may display a detailed virtual image 2451 including detailed information of the first virtual image 2450, fig. 24A).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Park171 to teach, generating a first partial image based on the first position and displaying the first partial image on the display surface in a case where the first position is located within a predetermined area for a predetermined duration, as suggested by Park421. The motivation would have been in order to prevent the user from “feeling dizzy or motion sick.”
As to claim 14, Park171 in view of Park421 teaches a non-transitory computer-readable storage medium storing a program that causes a computer to execute the control method according to claim 13 (Park171: [0089] the present disclosure may be implemented by an instruction which is stored in a non-transitory computer-readable storage medium (e.g., the memory 130 in FIG. 1) in the form of a program module. The instruction, if executed by a processor (e.g., the processor 120 in FIG. 1), may cause the one or more processors to execute the function corresponding to the instruction).
2. Claim(s) 4-5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Park et al (US 2018/0081171) in view of Park et al (US 2018/0005421) and further in view of Koyama (US 2020/0029111).
As to claim 4, Park171 in view of Park421 do not teach a playback image that has been recorded by a recorder as claimed.
However, Koyama teaches the control apparatus, wherein the omnidirectional image or hemispherical image is a playback image that has been recorded by a recorder ([0055] the image processing unit 24 generates a single VR image by implementing basic image processing on two images (fisheye images)… during playback, the image processing unit 24 executes image extraction processing, enlargement processing, distortion correction, and so on in order to display the VR image by VR display, and executes rendering for rendering the processing result in a predetermined storage area of a memory 32).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Park171 and Park421 to teach, wherein the omnidirectional image or hemispherical image is a playback image that has been recorded by a recorder, as suggested by Koyama. The motivation would have been in order to improve “operability when modifying a display region during playback of a moving image” ([0006]).
As to claim 5, Park171 in view of Park421 do not teach a live-view image captured by an image pickup apparatus as claimed.
However, Koyama teaches the control apparatus, wherein the omnidirectional image or hemispherical image is a live-view image captured by an image pickup apparatus ([0055] the image processing unit 24 generates a single VR image by implementing basic image processing on two images (fisheye images)… during VR display in the live view mode …).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Park171 and Park421 to teach, a live-view image captured by an image pickup apparatus, as suggested by Koyama. The motivation would have been in order to improve “operability when modifying a display region during playback of a moving image” ([0006]).
3. Claim(s) 9-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Park et al (US 2018/0081171) in view of Park et al (US 2018/0005421) and further in view of Schluessler et al (US 2018/0292895).
As to claim 9, Park171 in view of Park421 do not teach the control apparatus as claimed.
However, Schluessler teaches the control apparatus, wherein the processor can determine an intentional blink of the user ([0165] control a VR system based on eye blinking. For example, some embodiments may provide hands-free control to adjust resolution of a VR image using a detected voluntary blink. In some embodiments, a number of blinks (e.g. blink twice) may cause the system to enhance resolution of the VR image (e.g. or change the contrast or some other parameter or action)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Park171 and Park421 to teach, wherein the processor can determine an intentional blink of the user, as suggested by Schluessler. The motivation would have been in order to adjust “graphics rendering based on facial expression.” ([0001]). The invention improves user operability and user experience.
As to claim 10, Park171 in view of Park421 do not teach the control apparatus as claimed.
However, Schluessler teaches the control apparatus, wherein the processor is configured to: determine an intentional blink of the user, and switch the partial image in a case where a duration of the intentional blink of the user exceeds a predetermined duration ([0165] control a VR system based on eye blinking. For example, some embodiments may provide hands-free control to adjust resolution of a VR image using a detected voluntary blink. In some embodiments, a number of blinks (e.g. blink twice) may cause the system to enhance resolution of the VR image (e.g. or change the contrast or some other parameter or action)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Park171 and Park421 to teach, switching the partial image in a case where a duration of the intentional blink of the user exceeds a predetermined duration, as suggested by Schluessler. The motivation would have been in order to adjust “graphics rendering based on facial expression.” ([0001]). The invention improves user operability and user experience.
As to claim 11, Park171 in view of Park421 do not teach the control apparatus as claimed.
However, Schluessler teaches the control apparatus, wherein the processor is configured to: determine an intentional blink of the user, and switch the partial image in a case where the number of intentional blinks of the user exceeds a predetermined number of times ([0165] control a VR system based on eye blinking. For example, some embodiments may provide hands-free control to adjust resolution of a VR image using a detected voluntary blink. In some embodiments, a number of blinks (e.g. blink twice) may cause the system to enhance resolution of the VR image (e.g. or change the contrast or some other parameter or action)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Park171 and Park421 to teach, switching the partial image in a case where the number of intentional blinks of the user exceeds a predetermined number of times, as suggested by Schluessler. The motivation would have been in order to adjust “graphics rendering based on facial expression.” ([0001]). The invention improves user operability and user experience.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMEN W BOGALE whose telephone number is (571)270-1579. The examiner can normally be reached M-F 10:AM-6:PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Nitin Patel can be reached at (571)272-7677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMEN W BOGALE/Examiner, Art Unit 2628
/NITIN PATEL/Supervisory Patent Examiner, Art Unit 2628
Prosecution Insights