DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant's claim for foreign priority based on an application filed in CHINA on 19 May 2023. It is noted that the applicant has filed an interim copy of foreign priority on 16 March 2026. However, this does not constitute a certified copy of the CN202310573802.0 application. As a result, it is maintained that the applicant has not filed a certified copy of the CN202310573802.0 application as required by 37 CFR 1.55.
Response to Arguments
Applicant’s arguments with respect to rejections of claim(s) 1, 10, and 19 under 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2,5,10-11,14,19-20 rejected under 35 U.S.C. 103 as being unpatentable over YU; Zhixiong et al. (US 20180365797 A1) in view of TAKEDA; Takafumi et al. (US 20210090211 A1) in view of OGAWA; Kouji et al. (US 20190057496 A1)
Regarding claim 1, Yu teaches,
A method for panoramic video recording, (¶35,37, and 4, “image display method” run on a head-mounted display device using “fisheye or wide-angle lens” that collect panoramic or wide-angle pictures) comprising:
determining a spherical projection template (¶45 and fig. 4, Step 408 establishes “spherical model according to a horizontal field angle and a vertical field angle of a shooting lens” as disclosed in fig. 4) and an anti-distortion camera image (¶40-41 and fig. 4, “anti-distortion processing is restoring a shot distorted image”) at a current moment; (¶40-41 and 37, “anti-distortion processing” restoring shot distorted image using “a fisheye or wide-angle lens” which suffers “severe lens distortion”)
determining, for each spherical coordinate point (¶60 and fig. 9, “spherical surface of the spherical model”) within the spherical projection template, (¶60 and fig. 9, “spherical model” depicted in fig. 9) a mapping point texture of the spherical coordinate point (¶60 and fig. 9, “rectangular pyramid is lengthened along a radial direction of a spherical model, to intersect with a spherical surface of the spherical model” such that “a seen shape formed by borders of the spherical model should be the same as the shaped projection plane model”) within the anti-distortion camera image (¶60 and fig. 9, fill up the shaped cambered spherical model based on “shooting lens anti-distortion and spherization mapping” performed on the original distorted image) based on a field of view of a camera; (¶60,37, and fig. 9, “original distorted image” that suffers severe lens distortion collected using a “fisheye or wide-angle lens”) and
performing rendering on the spherical projection template (¶62-66 and fig. 4, Step 412 “stretch the original distorted image to fill up the shaped cambered spherical model”) based on the mapping point texture of each spherical coordinate point, (¶65 and 60, “spherization mapping processing are performed” according to the “rectangular pyramid lengthened along a radial direction of a spherical model, to intersect with a spherical surface of the spherical model” used to shape cambered spherical model) to obtain a panoramic video frame (¶62-65 and fig. 4, display “rendered image” after complete image rendering) recorded at the current moment. (¶62-65 and fig. 4, display “rendered image” that is an accurate and real-time depiction of “original distorted image”)
But does not explicitly disclose,
determining a two-dimensional spherical projection template
perform pixel rendering on the spherical projection template
However, Takeda teaches additionally,
determining a two-dimensional spherical projection template (¶177, “two-dimensional plane” obtained by “cutting out the area of a part of the spherical image” used to represent the “equirectangular projection image generated”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda which obtains a two-dimensional plane for a spherical image. This allows for display of an undistorted rectangular image.
Ogawa teaches additionally,
perform pixel rendering (¶79 and fig. 1, “composite image is generated” in accordance with “pixel of each captured image” equivalent/same position in the image of a “region in the spherical image” of a plurality of captured images mapped on the position corresponding to the position in the sphere) on the spherical projection template (¶79, pixels in a plurality of captured images mapped to position “corresponding to the position in the sphere in the composite image”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa which expressly maps pixel positions to corresponding positions in a spherical image. This allows for a technique that can generate composite images by continuously joining images without any image shifts.
Regarding claim 2, Yu with Takeda with Ogawa teach the limitations of claim 1,
Yu teaches additionally,
determining the mapping point texture of the spherical coordinate point (¶60 and fig. 9, “rectangular pyramid is lengthened along a radial direction of a spherical model, to intersect with a spherical surface of the spherical model” such that “a seen shape formed by borders of the spherical model should be the same as the shaped projection plane model”) within the anti-distortion camera image (¶60 and fig. 9, fill up the shaped cambered spherical model based on “shooting lens anti-distortion and spherization mapping” performed on the original distorted image) based on the field of view of the camera (¶60,37, and fig. 9, “original distorted image” that suffers severe lens distortion collected using a “fisheye or wide-angle lens”) comprises:
the anti-distortion camera image (¶40-41 and 37, “anti-distortion processing” restoring shot distorted image using “a fisheye or wide-angle lens” which suffers “severe lens distortion”)
Ogawa teaches additionally,
determining a pixel coordinate (¶102 and fig. 9, “coordinate values” of four corners indicating the “regions of the coordinate group”) of a mapping point (¶102 and fig. 9, “regions of the coordinate group”) of the spherical coordinate point (¶102 and fig. 9, coordinate values of the “specific region corresponding” to the specific imaging region in “the composite image”) based on the field of view of the camera (¶102,89, and fig. 9, “specific imaging region”, set to specific region corresponding to specific imaging region in the composite image, set according to “imaging region of the designated imaging unit”) and an established spherical coordinate mapping relationship; (¶102 and fig. 9, coordinate values of the “specific region corresponding to the specific imaging region in the composite image”) and
determining a corresponding mapping point texture within the camera image (¶136,89, and fig. 12, “determines whether the pixels of the image data” are pixels of the specific region of the “designated imaging unit as the specific imaging region”) based on the pixel coordinate of the mapping point. (¶136,102,89, and fig. 9, “regions of the coordinate group” associated with specific region corresponding to the specific imaging region designated with set according to “imaging region of the designated imaging unit”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa which expressly maps pixel positions to corresponding positions in a spherical image. This allows for a technique that can generate composite images by continuously joining images without any image shifts.
Regarding claim 5, Yu with Takeda with Ogawa teach the limitations of claim 1,
Yu teaches additionally,
acquiring an original camera image (¶36-37 and fig. 4, “original distorted image”) at the current moment; (¶36-37 and fig. 4, original distorted “image is collected” using a collecting device “camera” with a “fisheye or wide-angle lens”) and
performing anti-distortion processing (¶40-44 and fig. 4, step 406 “perform anti-distortion processing”) on the original camera image (¶40-44 and fig. 4, step 406 perform anti-distortion processing “restoring a shot distorted image”) to obtain a corresponding anti-distortion camera image. (¶44 and fig. 6, “image obtained after anti-distortion is performed” on an “original distorted image”)
Regarding claim 10, it is the electronic device claim of method claim 1.
Yu teaches additionally,
An electronic device (¶33-34 and fig. 2, “head-mounted display device” depicted in fig. 2) comprising:
a processor (¶33-34 and fig. 2, “graphic processing unit (GPU)” as part of an included processor and memory depicted in fig. 2) and a memory, (¶33-34 and fig. 2, “non-transitory computer readable storage medium” depicted in fig. 2) wherein the memory is configured to store a computer program, (¶33-34 and fig. 2, “non-transitory computer readable storage medium of the head-mounted display device stores an operating system”) and the processor is configured to invoke and run the computer program stored in the memory (¶33-34 and fig. 2, “processor is configured to provide computing and control capabilities” and “memory” providing an environment “running of the computer-readable instruction in the non-transitory computer readable storage medium” that implements an image display method)
Refer to rejection of claim 1 to teach the limitations of claim 10.
Regarding clam 11, dependent on claim 10, it is the electronic device of method claim 2, dependent on claim 1. Refer to rejection of claim 2 to teach the limitations of claim 11.
Regarding clam 14, dependent on claim 10, it is the electronic device of method claim 5, dependent on claim 1. Refer to rejection of claim 5 to teach the limitations of claim 14.
Regarding claim 19, it is the computer program claim of method claim 1.
Yu teaches additionally,
A non-transitory computer-readable storage medium storing a computer program, (¶33 and fig. 2, “non-transitory computer readable storage medium of the head-mounted display device stores an operating system” depicted in fig. 2) the computer program causing a computer (¶33 and fig. 2, “computer-readable instruction is executed to implement an image display method” depicted in fig. 2)
Refer to rejection of claim 1 to teach the limitations of claim 19.
Regarding clam 20, dependent on claim 19, it is the computer program claim of method claim 2, dependent on claim 1. Refer to rejection of claim 2 to teach the limitations of claim 19.
Claim(s) 3-4,12-13 rejected under 35 U.S.C. 103 as being unpatentable over YU; Zhixiong et al. (US 20180365797 A1) in view of TAKEDA; Takafumi et al. (US 20210090211 A1) in view of OGAWA; Kouji et al. (US 20190057496 A1) in view of POWELL; Karlton David (US 20210044725 A1)
Regarding claim 3, Yu with Takeda with Ogawa teach the limitations of claim 2,
Yu teaches additionally,
determining a corresponding pixel coordinate-angle representation relationship (¶55-61 and fig. 4, “countless rays are radiated from a spherical center”) based on a first pixel coordinate of a boundary vertex of a field of view in an imaging plane, (¶55-61 and fig. 4, “radiating countless rays from a spherical center” that connect the four edges of the shaped projection plane model to construct a “rectangular pyramid” to use as the “shaped cambered spherical model”) the first pixel coordinate being represented by the field of view of the camera, (¶55-61,34, and fig. 4, “radiating countless rays from a spherical center to penetrate four sides of a shaped projection plane model” to construct a “rectangular pyramid” which shares the aspect ratio with the “rectangular plane projection model” with a “shot picture”)
Ogawa teaches additionally,
determining a corresponding pixel (¶157, generate “specific region information” with “position information about objects and the imaging information”) coordinate-angle representation relationship (¶157, “view angle range” corresponding to the object) based on a second pixel coordinate of any space point of the field of view (¶157, “view angle range corresponding to the latitude range and the longitude range of the object” calculated in accordance with the lens information of the imaging information) in the imaging plane, (¶157, view angle range of the “object” set as “the specific imaging region of the object”) and the second pixel coordinate (¶157, “view angle range”) being represented by an angle between the space point of field of view and an optical axis of the camera; (¶157, “view angle range corresponding to the latitude range and the longitude range of the object” calculated in accordance with the lens information of the imaging information including “the latitude, the longitude, and the altitude of the imaging position”)
But does not explicitly teach the additional limitations of claim 3,
However, Powell teaches additionally,
determining a corresponding spherical coordinate-angle representation relationship (¶41, “spherical projection maps image sensor coordinates in the image sensor space to spherical coordinates” as depicted in fig. 4) based on a spherical coordinate of any space point of the field of view (¶41 and fig. 4, spherical distortion correction projection based on “pixel grid of the raw image is parameterized into corresponding physical dimensions” where the measured camera-specific optical center is “matched to the optical axis of the lens”) and an angle between the space point of the field of view (¶41 and fig. 4, “spherical coordinates” of the corrected image expressed in terms of “coordinates on the sphere represented by the azimuth arclength xs and the elevation arclength ys” as depicted in fig. 4) and the optical axis of the camera; (¶41 and fig. 4, “optic axis” at z-axis as depicted in fig. 4) and
determining a corresponding spherical coordinate mapping relationship (¶71-72, “data structure defining a matrix of pixels” each pixel including value (e.g., color/brightness/depth) corresponding with “pixel locations of the pixels of the raw image and the translated pixel locations of the pixels of the distortion corrected image 214”) based on the pixel coordinate-angle representation relationship and the spherical coordinate-angle representation relationship. (¶71-72,41, and fig. 4, “distortion corrected image 214” using position in the corrected image using “spherical distortion correction projection”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa with the spherical correction of Powell that matches the correction to an optical axis. This allows for distortion correction that is improved for the particular camera because of the matched optical axis.
Regarding claim 4, Yu with Takeda with Ogawa teach the limitations of claim 2,
Yu teaches additionally,
the spherical coordinate mapping relationship (¶45-54, “establishing a spherical model”) comprises a spherical coordinate horizontal mapping relational equation (¶45-54, “spherical model in a horizontal direction”) and a spherical coordinate vertical mapping relational equation, (¶45-54, spherical model in a “vertical direction”)
Ogawa teaches additionally,
the determining the pixel coordinate (¶102 and fig. 9, “coordinate values” of four corners indicating the “regions of the coordinate group”) of the mapping point (¶102 and fig. 9, “regions of the coordinate group”) of the spherical coordinate point (¶102 and fig. 9, coordinate values of the “specific region corresponding” to the specific imaging region in “the composite image”) based on the field of view of the camera (¶102,89, and fig. 9, “specific imaging region”, set to specific region corresponding to specific imaging region in the composite image, set according to “imaging region of the designated imaging unit”) and the established spherical coordinate mapping relationship (¶102 and fig. 9, coordinate values of the “specific region corresponding to the specific imaging region in the composite image”)
but does not explicitly teach the additional limitations of claim 4,
However, Powell teaches additionally,
substituting a horizontal field of view of the camera (¶41 and fig. 4, pixel grid of the raw image “shifted based on the measured camera-specific optical center” coordinates represented by x of the “(x, y) for the raw image” mapped to “spherical coordinates”) and an azimuth angle coordinate of the spherical coordinate point (¶41 and fig. 4, “azimuth arclength xs”) into the spherical coordinate horizontal mapping relational equation, (¶41 and fig. 4, “spherical coordinates” as a function in the form of “x(f, xs, ys, Rs)”) to obtain a horizontal pixel coordinate of the mapping point of the spherical coordinate point; (¶41 and fig. 4, function “x(f, xs, ys, Rs)” as of spherical projection for the X spherical distortion correction projection) and
substituting a vertical field of view of the camera, (¶41 and fig. 4, pixel grid of the raw image “shifted based on the measured camera-specific optical center” coordinates represented by y of the “(x, y) for the raw image” mapped to “spherical coordinates”) and the azimuth angle coordinate (¶41 and fig. 4, “azimuth arclength xs”) and an elevation angle coordinate of the spherical coordinate point (¶41 and fig. 4, “elevation arclength ys”) into the spherical coordinate vertical mapping relational equation, (¶41 and fig. 4, “spherical coordinates” as a function in the form of “y(f, xs, ys, Rs)”) to obtain a vertical pixel coordinate of the mapping point of the spherical coordinate point. (¶41 and fig. 4, function “y(f, xs, ys, Rs)” as of spherical projection for the Y spherical distortion correction projection)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa with the spherical correction of Powell that matches the correction to an optical axis. This allows for distortion correction that is improved for the particular camera because of the matched optical axis.
Regarding clam 12, dependent on claim 11, it is the electronic device of method claim 3, dependent on claim 2. Refer to rejection of claim 3 to teach the limitations of claim 12.
Regarding clam 13, dependent on claim 11, it is the electronic device of method claim 4, dependent on claim 2. Refer to rejection of claim 4 to teach the limitations of claim 13.
Claim(s) 6-8,15-17 rejected under 35 U.S.C. 103 as being unpatentable over YU; Zhixiong et al. (US 20180365797 A1) in view of TAKEDA; Takafumi et al. (US 20210090211 A1) in view of OGAWA; Kouji et al. (US 20190057496 A1) in view of TAKAHASHI; Ryohei et al. (US 20190347760 A1)
Regarding claim 6, Yu with Takeda with Ogawa teaches the limitations of claim 1,
Ogawa teaches,
determining the mapping point texture of the spherical coordinate point (¶126-127 and fig. 11, step ST25 “determines whether there is the corresponding texture”) within the anti-distortion camera image (¶126-127, “corresponding texture” having the same time code as that of the composite image) based on the field of view of the camera; (¶126-127 and fig. 11, ST26 “combines the texture having the same time code as that of the composite image with an image of the specific region” when there is the corresponding texture) and
using a preset texture as the mapping point texture of the second-class spherical coordinate point; (¶126-128, move to step ST27 “performs an image output process” where “image data of the composite image to a display device” when there is no corresponding texture)
But does not explicitly teach the additional limitations of claim 6,
However, Takahashi teaches additionally,
determining an optimal resolution of the spherical projection template (¶76-80, “celestial sphere image” projected to “projection structure to obtain a projected frame” to obtain a “region-wise” packed frame) based on the field of view of the camera, (¶80 and fig. 1, “projected frame 14” with six faces (regions) including a front face, a right face, a back face, a left face, a top face, and a bottom face as depicted in fig. 1) a camera resolution, (¶80,140,75, fig. 1 and 18, six faces (regions) including a front face, a right face, a back face, a left face, a top face, and a bottom face with resolutions such that “region image of each face have 400x400 pixels” depicted in fig. 18, photographed by a camera 1 depicted in fig. 1) and a rendering field of view (¶79-80, performing region-wise packing that attains “packed frame 16”) of the spherical projection template; (¶79-80, packed frame 16 obtained from “projected frame 14” expressing the celestial sphere image 2-dimensionally)
determining first-class spherical coordinate points (¶80,142, and fig. 18-B, “resolution of the region image of the front face (front)” with a resolution of “(400x400)” depicted in fig. 18) and second-class spherical coordinate points (¶80,142, and fig. 18-B, “resolutions of the other region images” reduced to “½ (200×200) horizontally and vertically” depicted in fig. 18) within the spherical projection template based on the optimal resolution; (¶80,142,76, and fig. 18, “packed frame 411 generated from the projected frame 401” based on “region-wise packing” that optimizes transmission capacity by adjusting resolution)
determining, for each of the first-class spherical coordinate points, (¶80,142, and fig. 18-B, “region image of the front face (front)” depicted in fig. 18) the mapping point texture of the first-class spherical coordinate point (¶142 and fig. 18, “front face (front) is considered to be (400×400) as it stands”) within the camera image based on the field of view of the camera; (¶142 and 79, “region image of the front face (front)” included in “images of six faces (regions)”) and
using, for each of the second-class spherical coordinate points, (¶80,142, and fig. 18-B, “other region images” including upper side and the lower side of the left face (left), the right face (right), and the back face (back) depicted in fig. 18) a preset texture as the mapping point texture (¶142 and 140, “other region images are reduced to ½ (200×200) horizontally and vertically” from the original “region image of each face have 400×400 pixels”) of the second-class spherical coordinate point; (¶142 and 140, “other region images”)
wherein the first-class spherical coordinate points (¶80,142, and fig. 18-B, “resolution of the region image of the front face (front)” with a resolution of “(400x400)” depicted in fig. 18) are spherical coordinate points (¶79-80,142, and fig. 18-B, “front face (front)”) within the field of view range of the camera within the spherical projection template. (¶80,142,75-76, and fig. 18, “packed frame 411 generated from the projected frame 401” based on “region-wise packing” that optimizes transmission capacity by adjusting resolution of omnidirectional image (celestial sphere image) “photographed by a camera 1)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa with the resolution adjust resolution of Takahashi which adjusts the packing of a sphere image. This allows for optimized transmission capacities that factors preference and what images only need to be sufficient.
Regarding claim 7, Yu with Takeda with Ogawa with Takahashi teaches the limitations of claim 1,
Takahashi teaches additionally,
determining the first-class spherical coordinate points (¶80,142, and fig. 18-B, “resolution of the region image of the front face (front)” with a resolution of “(400x400)” depicted in fig. 18) and the second-class spherical coordinate points (¶80,142, and fig. 18-B, “resolutions of the other region images” reduced to “½ (200×200) horizontally and vertically” depicted in fig. 18) within the spherical projection template based on the optimal resolution (¶80,142,76, and fig. 18, “packed frame 411 generated from the projected frame 401” based on “region-wise packing” that optimizes transmission capacity by adjusting resolution) comprises:
determining the spherical coordinate points (¶79-80 and fig. 1, “six faces (regions)” disposed in the projected frame 14 depicted in fig. 1) within the spherical projection template (¶79-80,75, and fig. 1, “celestial sphere image” expressed 2-dimensionally as “images of six faces (regions)”) based on the optimal resolution of the spherical projection template; (¶76,79, and fig. 1, selectively adjusting images of six faces (regions) “resolution of a region in which high quality is preferable and decreasing a resolution of a region in which low quality is sufficient”)
selecting the spherical coordinate points (¶80 and fig. 1, “region image of the front face (front)” region-wise packed into packed frame 16 depicted in fig. 1) within the field of view range of the camera (¶79-80,75, and fig. 1, “front face (front)” of the images of six faces (regions) “photographed by a camera 1” as a celestial sphere image) from the spherical coordinate points (¶79-80, “six faces (regions)”) within the spherical projection template (¶79-80,75, and fig. 1, “celestial sphere image” expressed 2-dimensionally as “images of six faces (regions)”) as the first-class spherical coordinate points; (¶79-80 and fig. 1, “resolution of the region image of the front face (front) increases” of the images of six faces (regions)) and
using remaining spherical coordinate points (¶79-80 and fig. 1, “other region images” of the images of six faces (regions)) other than the first-class spherical coordinate points (¶79-80 and fig. 1, “other region images” separate from the region image of the front face (front) of the “images of six faces (regions)”) within the spherical projection template (¶79-80,75, and fig. 1, “celestial sphere image” expressed 2-dimensionally as “images of six faces (regions)”) as the second-class spherical coordinate points. (¶79-80 and fig. 1, “resolutions of the other region images remain unchanged” of the images of six faces (regions))
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa with the resolution adjust resolution of Takahashi which adjusts the packing of a sphere image. This allows for optimized transmission capacities that factors preference and what images only need to be sufficient.
Regarding claim 8, Yu with Takeda with Ogawa with Takahashi teaches the limitations of claim 6,
Takahashi teaches additionally,
wherein the determining the first-class spherical coordinate points (¶80,142, and fig. 18-B, “resolution of the region image of the front face (front)” with a resolution of “(400x400)” depicted in fig. 18) and the second-class spherical coordinate points (¶80,142, and fig. 18-B, “resolutions of the other region images” reduced to “½ (200×200) horizontally and vertically” depicted in fig. 18) within the spherical projection template based on the optimal resolution (¶80,142,76, and fig. 18, “packed frame 411 generated from the projected frame 401” based on “region-wise packing” that optimizes transmission capacity by adjusting resolution) comprises:
converting the spherical projection template (¶79-80 and fig. 1, “performing region-wise packing on the projected frame 14” expressing the celestial sphere image 2-dimensionally to obtain “packed frame 16” depicted in fig. 1) into a first spherical projection template (¶79-80, region-wise packing “front face (front)” of projected frame 14 including images of six faces (regions)) and a second spherical projection template, (¶79-80, region-wise packing “other region images” of projected frame 14 including images of six faces (regions)) a rendering field of view (¶79-80 and 142, region-wise packed “front face (front)” region image) and an image resolution (¶79-80 and 142, “front face (front) is considered to be (400×400)”) of the first spherical projection template (¶79-80 and 142, “front face (front)” region image of the images of six faces (regions)) being the field of view of the camera (¶79-80,142, and 75, “front face (front)” region image that expresses “celestial sphere image 2-dimensionally” of the omnidirectional image (celestial sphere image) photographed by a camera 1) and the camera resolution, respectively, (¶79-80,142,140 and 75, “front face (front) is considered to be (400×400) as it stands” formed in the projected frame 401 that expresses the celestial sphere image 2-dimensionally photographed by camera 1) and a rendering field of view (¶79-80 and 142, region-wise packed “other region images” of the ) and an image resolution (¶79-80 and 142, “other region images are reduced to ½ (200×200) horizontally and vertically”) of the second spherical projection template (¶79-80 and 142, “other region images” of the images of six faces (regions)) being the rendering field of view of the spherical projection template (¶79-80,142, and 75, “other region images” that express “celestial sphere image 2-dimensionally” of the omnidirectional image (celestial sphere image) photographed by a camera 1) and the optimal resolution of the spherical projection template, respectively; (¶79-80,142,140 and 75-76, “other region images are reduced to ½ (200×200) horizontally and vertically”, in which low quality is sufficient, formed in the projected frame 401 that expresses the celestial sphere image 2-dimensionally photographed by camera 1)
using spherical coordinate points (¶76,142, and fig. 18, adjusting according to “the position or the size” for the “front face (front)” region for packing from the projected frame 401 into a packed frame 411 depicted in fig. 18) within the first spherical projection template (¶79-80 and 142, “front face (front)” region image of the images of six faces (regions)) as the first-class spherical coordinate points (¶80,142, and fig. 18-B, “resolution of the region image of the front face (front)” with a resolution of “(400x400)” depicted in fig. 18) based on the image resolution of the first spherical projection template; (¶79-80,142,140 and 75, “front face (front) is considered to be (400×400) as it stands” formed in the projected frame 401 that expresses the celestial sphere image 2-dimensionally photographed by camera 1) and
using spherical coordinate points (¶76,142, and fig. 18, adjusting according to “the position or the size” for the “other region images” for packing from the projected frame 401 into a packed frame 411 depicted in fig. 18) within the second spherical projection template (¶79-80 and 142, “other region images” of the images of six faces (regions)) as the second-class spherical coordinate points (¶80,142, and fig. 18-B, “resolutions of the other region images” reduced to “½ (200×200) horizontally and vertically” depicted in fig. 18) based on the image resolution of the second spherical projection template. (¶79-80,142,140 and 75-76, “other region images are reduced to ½ (200×200) horizontally and vertically”, in which low quality is sufficient, formed in the projected frame 401 that expresses the celestial sphere image 2-dimensionally photographed by camera 1)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa with the resolution adjust resolution of Takahashi which adjusts the packing of a sphere image. This allows for optimized transmission capacities that factors preference and what images only need to be sufficient.
Regarding clam 15, dependent on claim 10, it is the electronic device of method claim 6, dependent on claim 1. Refer to rejection of claim 6 to teach the limitations of claim 15.
Regarding clam 16, dependent on claim 15, it is the electronic device of method claim 7, dependent on claim 6. Refer to rejection of claim 7 to teach the limitations of claim 16.
Regarding clam 17, dependent on claim 15, it is the electronic device of method claim 8, dependent on claim 6. Refer to rejection of claim 8 to teach the limitations of claim 17.
Claim(s) 9,18 rejected under 35 U.S.C. 103 as being unpatentable over YU; Zhixiong et al. (US 20180365797 A1) in view of TAKEDA; Takafumi et al. (US 20210090211 A1) in view of OGAWA; Kouji et al. (US 20190057496 A1) in view of TAKAHASHI; Ryohei et al. (US 20190347760 A1) in view of Sano; Genjiro (US 20200260004 A1)
Regarding claim 9, Yu with Takeda with Ogawa with Takahashi teaches the limitations of claim 8,
But does not explicitly teach the additional limitations of claim 9,
However, Sano teaches additionally,
performing pixel rendering on the first spherical projection template (¶35, “image processing unit 24 converts” one of the two images “to entire-celestial-sphere-form (entire-celestial-sphere-image-form) images by correcting the distortion in the images by geometric transformation”) based on the mapping point texture of each of the first-class spherical coordinate points, (¶35, “correcting the distortion in the images by geometric transformation by taking the detected connecting position and lens characteristics of the optical systems into consideration”) to obtain a first panoramic candidate frame at the current moment; (¶35, “converts” one of the two images “to entire-celestial-sphere-form (entire-celestial-sphere-image-form) images”)
performing pixel rendering on the second spherical projection template (¶35, “image processing unit 24 converts” the second of the two images “to entire-celestial-sphere-form (entire-celestial-sphere-image-form) images by correcting the distortion in the images by geometric transformation”) based on the mapping point texture of each of the second-class spherical coordinate points, (¶35, “correcting the distortion in the images by geometric transformation by taking the detected connecting position and lens characteristics of the optical systems into consideration”) to obtain a second panoramic candidate frame at the current moment; (¶35, “converts” second of the two images “to entire-celestial-sphere-form (entire-celestial-sphere-image-form) images”) and
merging the first panoramic candidate frame and the second panoramic candidate frame, (¶35, “combining (blending) the two entire-celestial-sphere-form images”) to obtain the panoramic video frame recorded at the current moment. (¶35, “image processing unit 24 generates one entire-celestial sphere image (VR image) by combining (blending) the two entire-celestial-sphere-form images”)
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention to combine the image display of Yu with the with the rendering of Takeda with the composite image generation of Ogawa with the resolution adjust resolution of Takahashi with the image processing of Sano which blends two entire-celestial-sphere-form images. Using this teaching can help improve the appearance of seams in the entire-celestial sphere image and suppress feeling that something is wrong.
Regarding clam 18, dependent on claim 17, it is the electronic device of method claim 9, dependent on claim 8. Refer to rejection of claim 9 to teach the limitations of claim 18.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIMMY S LEE whose telephone number is (571)270-7322. The examiner can normally be reached Monday thru Friday 10AM-8PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph G. Ustaris can be reached at (571) 272-7383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSEPH G USTARIS/Supervisory Patent Examiner, Art Unit 2483
/JIMMY S LEE/Examiner, Art Unit 2483
Prosecution Insights