Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 9-13 and 15-20 have been considered but are not persuasive. Roulet is relied upon for portions of the newly amended subject matter, Roulet is does not teach many of the new features amended into the claims. The examiner relies on a combination of Roulet and Chou US 2018/0027181. Since applicant’s arguments do not specifically address the new grounds of rejection comprising the combination of Roulet and Chou, they are unpersuasive. Claims 2-9 remain objected for depending from a rejected base claim but contain allowable subject matter and new claims 21 and 22 are not rejected herein.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 9-13, 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over in view of Roulet US 2018/0027181 and Chou US 20170280141 A1 in view of Shen US 2019/0379910.
A fisheye image compression method comprising:
acquiring positioning information of a render region of a decoding end; (see paragraph 37 note that a part of the panoramic image which is to be displayed is determined note the display is at the decoder end see figure 8 and or figure 2);
identifying the render region on at least two fisheye images to identify a fisheye render region in the at least two fisheye image images according to the positioning information, wherein the region in the at least two fisheye image images apart from the fisheye render region is a fisheye non-render region; (see figure 7 note that render region comprises portion of both fisheye lenses and paragraph 34 note that the parts of the native panoramic images which are required for assembly are determined also see paragraph 37 note that parts of the image being assembled are communicated to the decoder see paragraph 31 “ These camera modules can be any image capture devices such as fisheye, panomorph, wide-angle, traditional narrow angle, light field or plenoptic camera modules.” Note that the camera images which may be captured from fisheye lenses and are therefore fisheye images)
and compressing the at least two fisheye images to obtain a compressed image; (see figure 8 and paragraph 35 The encoding unit 850 may simply write the information (metadata) 830 and native panoramic image 820 into a file or may perform more intensive tasks such as compressing or encrypting the resulting file”);
wherein the identifying the render region onto at least two fisheve images to identify the fisheye render region according to the positioning information comprises: (see figure 7 note that render region comprises portion of both fisheye lenses and paragraph 34 note that the parts of the native panoramic images which are required for assembly are determined also see paragraph 37 note that parts of the image being assembled are communicated to the decoder see paragraph 31 “ These camera modules can be any image capture devices such as fisheye, panomorph, wide-angle, traditional narrow angle, light field or plenoptic camera modules.” Note that the camera images which may be captured from fisheye lenses and are therefore fisheye images)
identify a second two-dimensional point set, wherein the second two-dimensional point set is distributed across at least two fisheve images; (see paragraph 34 and figure 7 note that the portions of regions of images 710 and 720 comprise a two dimensional point sets)
onto at least two fisheye images identifying the fisheve render region of the at least two fisheye images according to the second two-dimensional point set. (see paragraph 34 and figure 7 note that the portions of regions 710 and 720 comprise a two dimensional point sets see also paragraph 37 note that the parts of the fish eye images required from the native image is communicated from the decoder to the encoder)
Roulet does not disclose projecting the region, obtaining a first two-dimensional point set according to the positioning information, wherein the first two-dimensional point set is present in the render region;
projecting the first two-dimensional point set to determine a render region in a spherical image
or
wherein a compression ratio of the fisheye render region in the compressed image is less than a compression ratio of the fisheye non-render region therein, and/or the fisheye render region is not compressed,
Chou discloses
Projecting the region (see paragraph 41 note that a 2d portion of the 2d frame is projected onto a spherical image)
obtaining a first two-dimensional point set according to the positioning information, wherein the first two-dimensional point set is present in the render region; (see paragraph 41 note that a 2d portion of a rendered frame is projected onto a spherical image the 16X16 block of pixels corresponds first set of points)
projecting the first two-dimensional point set to determine a render region in a spherical image (see paragraph 41 note that the two-dimensional set of points is projected on to the spherical image)
The motivation to combine is “For example, regions that exhibit higher distortion will be allocated a bitrate that is higher than the average frame rate (e.g., 3 Mbps), whereas regions in the 360 video that exhibit less distortion are allocated a bitrate that is lower than the average frame rate” One of ordinary skill the art could have easily use the projection method of Chou to determine the rendered portion of figure 7 of Roulet comprising portions of the two fisheye images. Therefore, it would have been obvious before the effective filing date of the claimed invention to combine Chou and Roulet to reach the aforementioned advantage.
Roulet does not disclose wherein the compression ratio of the indicated region in the compressed image is less than the compression ratio of the fisheye non-indicated region therein, and/or the fisheye indicated region is not compressed. Shen discloses wherein the compression ratio of the selected region in the compressed image is less than the compression ratio of the fisheye non-selected region therein, and/or the fisheye selected region is not compressed. (see paragraph 53 and 54 note that a focus region is compressed at a higher resolution than a non-focus region). The motivation to combine is to leave an important region uncompressed while compressing the less important region (see paragraph 53 and 54). One of ordinary skill in the art could have easily compressed the non-render region of Roulet more that of the render region using the teaching of Shen. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Roulet Chou and Shen to reach the aforementioned advantage.
Re claim 9 Roulet further discloses fisheye images and an indicated render region (see rejection to claim 1) but does not expressly disclose
wherein the step of compressing the image to obtain a compressed image comprises: generating at least one down-sampling mapping table, each of which records mapping relationships between the non-indicated region and the compressed image, as well as mapping relationships between the indicated region and the compressed image;
and remapping the non-indicated region and the indicated render region of the fisheye image according to each of the at least one down-sampling mapping table to obtain the compressed image
Shen further discloses
wherein the step of compressing the image to obtain a compressed image comprises: generating at least one down-sampling mapping table, each of which records mapping relationships between the non-indicated region and the compressed image, as well as mapping relationships between the indicated region and the compressed image; (see paragraph 49 note that an encoding table is used to map pixel position in the image during the encoding see paragraph 54 note that the compressed image has been down sampled, note that the look up table is used for each pixel so it will have relationships for all pixels in the image including both regions ).
and remapping the non-indicated region and the indicated render region of the fisheye image according to each of the at least one down-sampling mapping table to obtain the compressed image. (see paragraph 49 note that an encoding table is used to map pixel position in the image during the encoding see paragraph 54 note that the encoded image has been down sampled, note that the look up table is used for each pixel so it will have relationships for all pixels in the image including both regions).
The motivation to combine reduce calculation (see paragraph 49). One of ordinary skill in the art could have easily used a look up table to position the pixels compressed by the method of Shen in the method of Roulet. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention combine Roulet CHou and Shen reach the aforementioned advantage.
Re claim 10 Roulet discloses fisheye video stream compression method comprising: acquiring a fisheye video stream (see paragraph 31 note that video images are captured); compressing each frame of fisheye image of the fisheye video stream to obtain a compressed image for each frame of fisheye image by using the fisheye image compression method (see paragraph 35 and figure 8 note that the captured images are compressed by the encoder) according claim 1 (see rejection to claim 1) to obtain a compressed fisheye video stream (see paragraph 35 and 36 note that the stream encoded may be an image or video)
Roulet and Chou do not expressly disclose performing video stream compression based on the compressed image of each frame of the video stream.
Shen further discloses performing video stream compression based on the compressed image of each frame of the video stream. (see paragraph 80 note that interframe compression may be used to compress the video stream as an additional function). The motivation to combine is to further compress the video stream (See paragraph 80). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Shen Chou and Roulet to reach the aforementioned advantage.
Re claim 11 Roulet discloses A panoramic video generation method comprising: acquiring a compressed fisheye video stream; wherein the compressed fisheye video stream is processed by the fisheye video stream compression method according to Claim 10; (see rejection to claim 10)
obtaining a compressed image corresponding to multiple frames of original fisheye images according to the compressed fisheye video stream; restoring the compressed image to obtain the original fisheye images; (see paragraph 36 and figure 2 and figure 8 note that the decoder receives the encoded video of fisheye images and decodes [restores])
splicing the original fisheye images together to obtain a panoramic video (see paragraph 34 and figure 7 note that portions of the images are extracted and assembled to form the images from the video).
Re claim 12 Roulet further discloses fisheye images and an indicated render region (see rejection to claim 1) but does not expressly disclose herein the step of restoring the compressed images to obtain the original images comprises: analyzing the compressed image to obtain original positioning information in the original image of each pixel point in the indicated region; decoding the compressed image according to the original positioning information to recover the original image. Shen Further disclose original images comprises: analyzing the compressed image to obtain original positioning information in the original image of each pixel point; decoding the compressed image according to the original positioning information to recover the original image (see paragraph 52 note that a one step decoding table is used to decoder each pixel in the image (including the indicated region) so that they can be displayed see also paragraph 70). The motivation to combine is to allow for one step decoder (see paragraph 52). One of ordinary skill in the art could have easily used a look up table to position the pixels compressed by the method of Shen in the method of Roulet. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention combine Roulet Chou and Shen reach the aforementioned advantage.
Re claim 13 Roulet further discloses fisheye images and a indicated render region (see rejection to claim 1 ) but does not expressly disclose wherein the step of restoring the compressed images to obtain the original images comprises: obtaining a reverse mapping table generated according to a down-sampling mapping table (see paragraph 52 note that decoding table is used to map pixel position in the decoder back to the original position to reconstruct the spherical video see paragraph 54 note that the compressed image has been down sampled ); remapping the compressed image according to the reverse mapping table to obtain the original fisheye image (see paragraph 52 note that decoding table is used to map pixel position in the decoder back to the original position to reconstruct the spherical video see paragraph 54 note that the compressed image has been down sampled ). The motivation to combine is to allow for one step decoder (see paragraph 52). One of ordinary skill in the art could have easily used a look up table to position the pixels compressed by the method of Shen in the method of Roulet. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention combine Roulet Chou and Shen reach the aforementioned advantage.
Re claim 15 Roulet and Shen Chou discloses the steps of fisheye image compression method according to claim 1 (see rejection to claim 1). Roulet and Shen further do not expressly disclose A computer apparatus comprising a memory and a processor, a computer program being stored in the memory, wherein the processor implements the method, when executing the computer program. Chou discloses A computer apparatus comprising a memory and a processor, a computer program being stored in the memory, wherein the processor implements the method, when executing the computer program (see paragraph 26 27 and 29 note that functions may be implemented by a memory storing code and processor executing the code). One of ordinary skill in the art could have easily implemented the steps of Roulet and Shen with a memory storing software programs and a computer processor and the results would be the same and therefore predictable. The elements of the various reference do the same thin separately as the do together and the computer and memory as combined is just a tool for carrying out the function of the references. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chou with Roulet and Shen.
Re claim 17 Roulet and Shen discloses the steps of fisheye image compression method according to claim 10 (see rejection to claim 10). Roulet and Shen further do not expressly disclose A computer apparatus comprising a memory and a processor, a computer program being stored in the memory, wherein the processor implements the method, when executing the computer program. Chou discloses A computer apparatus comprising a memory and a processor, a computer program being stored in the memory, wherein the processor implements the method, when executing the computer program (see paragraph 26 27 and 29 note that functions may be implemented by a memory storing code and processor executing the code). One of ordinary skill in the art could have easily implemented the steps of Roulet and Shen with a memory storing software programs and a computer processor and the results would be the same and therefore predictable. The elements of the various reference do the same thin separately as the do together and the computer and memory as combined is just a tool for carrying out the function of the references. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chou with Roulet and Shen.
Re claim 18 Roulet and Shen discloses the steps of fisheye image compression method according to claim 11 (see rejection to claim 11). Roulet and Shen further do not expressly disclose A computer apparatus comprising a memory and a processor, a computer program being stored in the memory, wherein the processor implements the method, when executing the computer program. Chou discloses a computer apparatus comprising a memory and a processor, a computer program being stored in the memory, wherein the processor implements the method, when executing the computer program (see paragraph 26 27 and 29 note that functions may be implemented by a memory storing code and processor executing the code). One of ordinary skill in the art could have easily implemented the steps of Roulet and Shen with a memory storing software programs and a computer processor and the results would be the same and therefore predictable. The elements of the various reference do the same thin separately as the do together and the computer and memory as combined is just a tool for carrying out the function of the references. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chou with Roulet and Shen.
Re claim 16 Roulet and Shen discloses the steps of fisheye image compression method according to claim 1 (see rejection to claim 1). Roulet and Shen further do not expressly disclose a computer-readable non-volatile storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method. Chou discloses a computer-readable non-volatile storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method (see paragraph 26 27 and 29 note that functions may be implemented by a memory storing code and processor executing the code). One of ordinary skill in the art could have easily implemented the steps of Roulet and Shen with a memory storing software programs and a computer processor and the results would be the same and therefore predictable. The elements of the various reference do the same thin separately as the do together and the computer and memory as combined is just a tool for carrying out the function of the references. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chou with Roulet and Shen.
Re claim 19 Roulet and Shen discloses the steps of fisheye image compression method according to claim 10 (see rejection to claim 10). Roulet and Shen further do not expressly disclose A computer-readable non-volatile storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method. Chou discloses a computer-readable non-volatile storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method (see paragraph 26 27 and 29 note that functions may be implemented by a memory storing code and processor executing the code). One of ordinary skill in the art could have easily implemented the steps of Roulet and Shen with a memory storing software programs and a computer processor and the results would be the same and therefore predictable. The elements of the various reference do the same thin separately as the do together and the computer and memory as combined is just a tool for carrying out the function of the references. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chou with Roulet and Shen.
Re claim 20 Roulet and Shen discloses the steps of fisheye image compression method according to claim 11 (see rejection to claim 11). Roulet and Shen further do not expressly disclose A computer-readable non-volatile storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method. Chou discloses A computer-readable non-volatile storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method (see paragraph 26 27 and 29 note that functions may be implemented by a memory storing code and processor executing the code). One of ordinary skill in the art could have easily implemented the steps of Roulet and Shen with a memory storing software programs and a computer processor and the results would be the same and therefore predictable. The elements of the various reference do the same thin separately as the do together and the computer and memory as combined is just a tool for carrying out the function of the references. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Chou with Roulet and Shen.
Allowable Subject Matter
Claim 2-8 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Claims 21 and 22 are allowed.
Re claim 2 Shen discloses in paragraph 49 “Rather than perform calculations for each pixel, lookup tables may be used to read the results for various input pixel locations. First lookup table 202 receives a pixel location (λ, ϕ) and outputs that pixel's (x,y) location after moving lower-right region 172 and lower-left region 170 to the non-effective area in the top corners. For pixels within the effective area of the input image, including lower-left region 170 and lower-right region 172, sinusoidal mapping is performed to convert the pixel's (λ, ϕ) location to outputs that pixel's (x,y) location within rectangular bounding image 180.”
The prior art of record does not expressly disclose the combination of “wherein the step of identifying a fisheye render region in a fisheye image according to the positioning information comprises: collecting a plurality of points at equal intervals on a boundary of the render region to obtain a first two-dimensional point set; projecting the first two-dimensional point set onto a spherical coordinate system according to the positioning information of the render region to obtain a three-dimensional point set; projecting the three-dimensional point set onto the fisheye image corresponding to the positioning information to obtain a second two-dimensional point set; and identifying the fisheye render region of the fisheye image according to the second two-dimensional point set.”
Re claims 3-4 these claims depend from claim 2.
Re claim 5 Shen further discloses “Input spherical image 222 is divided into 16 blocks, labeled 1 to 16 in FIG. 13B. Focus area 220 is in blocks 8 and 11. The focus blocks remain at the full input resolution, but other non-focus blocks 1-7, 9-10, and 12-16 are downsampled by one-half in both x and y directions, so that the downsampled blocks require only one-quarter the space as the original full-resolution blocks” in paragraph 54.
The prior art of record does not expressly disclose “calculating an area of the compressed image according to a preset compression ratio and the resolution of the fisheye image; down-sampling the fisheye image according to a first compression ratio to obtain a fisheye thumbnail; and when the sum of pixel point numbers in the fisheye thumbnail and the fisheye render region is less than or equal to the sum of pixel point numbers in the compressed image, storing pixel points in the fisheye render region and pixel points in the fisheye thumbnail into the compressed image.”
Re claims 6-8 these claims depend from claim 5.
Claim 21 The allowable subject matter of this claim is the same as claim 5.
Re claim 22 this claim depends from claim 21.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN T MOTSINGER whose telephone number is (571)270-1237. The examiner can normally be reached 9AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN T MOTSINGER/Primary Examiner, Art Unit 2673
Prosecution Insights