DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1-15 are objected to because of the following informalities:
For claim 1, Examiner believes this claim should be amended in the following manner:
An image processing method, comprising:
receiving a first image sequence transmitted based on a first transmission frame rate and a second image sequence transmitted based on a second transmission frame rate, wherein the first image sequence comprises a first image; the second image sequence comprises second images; image content of each of the second images is [[the]] same as image content of a first area in the first image; and the first transmission frame rate is less than the second transmission frame rate;
generating a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence comprises third images; the third images are in a one-to-one correspondence with the second images; image content of the first area in each of the third images comes from the second images; and image content of areas, other than the first area, in the third images comes from the first image; and
generating a video stream based on the third image sequence and transmitting the video stream to a user.
For claim 2, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 1, wherein the generating [[a]] the third image sequence based on the first image sequence and the second image sequence comprises:
augmenting the first image sequence based on the second transmission frame rate, wherein [[the]] first images in the augmented first image sequence are in a one-to-one correspondence with the second images;
replacing the image content of the first area in [[the]] each first image with [[the]] each corresponding second image, wherein the first images are the third images; and
generating the third image sequence based on the third images.
For claim 3, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 2, wherein the replacing the image content of the first area in [[the]] each first image with [[the]] each corresponding second image comprises:
determining a coordinate correspondence between pixel points in [[the]] each corresponding second image and pixel points in the first area in [[the]] each first image based on a width and height of [[the]] each first image, a width and height of the first area, a spherical angle range covered by the first area, and coordinates of a center point of a video picture of the user; and
replacing the pixel points in the first area in [[the]] each first image with the pixel points in [[the]] each corresponding second image based on the coordinate correspondence.
For claim 4, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 1, wherein the second image sequence further comprises a fourth image; image content of the fourth image is [[the]] same as image content of a second area in the first image; and the second area is located in areas other than the first area; and the image processing method further comprises:
determining whether to update the first image, based on a similarity between the image content of the fourth image and the image content in the second area in the first image; and if so, updating the first image.
For claim 5, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 1, wherein after transmitting the video stream to the user, the image processing method further comprises:
updating coordinates of a center point of the first area in the first image based on [[the]]] coordinates of [[the]] a center point of [[the]] a video picture of the user when watching the video stream; and
updating the first area in the first image based on the updated coordinates of the center point of the first area.
For claim 6, Examiner believes this claim should be amended in the following manner:
An image processing method, comprising:
transmitting a first image sequence based on a first transmission frame rate, and transmitting a second image sequence based on a second transmission frame rate, wherein the first image sequence comprises a first image; the second image sequence comprises second images; image content of each of the second images is [[the]] same as image content of a first area in the first image; and the first transmission frame rate is less than the second transmission frame rate.
For claim 7, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 6, wherein before transmitting the first image sequence based on the first transmission frame rate, and transmitting the second image sequence based on the second transmission frame rate, the image processing method further comprises:
capturing the first image according to a preset capture frame rate;
determining the first area in the first image; and
cropping the first area in the first image, to obtain the second images.
For claim 8, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 7, wherein the determining the first area in the first image comprises:
determining the first area in the first image based on a difference in image content of a plurality of first images with adjacent timestamps.
For claim 9, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 7, wherein the cropping the first area in the first image, to obtain the second images comprises:
rotating the first image based on [[the]] coordinates of [[the]] a center point of the first area, wherein the coordinates of the center point of the first area in the [[rotated]] first image as rotated are located at a center of the [[rotated]] first image as rotated; and
cropping the first area in the [[rotated]] first image as rotated, to obtain the second images.
For claim 10, Examiner believes this claim should be amended in the following manner:
The image processing method according to claim 6, wherein after transmitting the first image sequence based on the first transmission frame rate, and transmitting the second image sequence based on the second transmission frame rate, the image processing method further comprises:
obtaining [[the]] updated coordinates of [[the]] a center point of the first area; and updating the first area in the first image based on the updated coordinates of the center point of the first area.
For claim 11, Examiner believes this claim should be amended in the following manner:
An electronic device, comprising:
a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to:
receive a first image sequence transmitted based on a first transmission frame rate and a second image sequence transmitted based on a second transmission frame rate, wherein the first image sequence comprises a first image; the second image sequence comprises second images; image content of each of the second images is [[the]] same as image content of a first area in the first image; and the first transmission frame rate is less than the second transmission frame rate;
generate a third image sequence based on the first image sequence and the second image sequence, wherein the third image sequence comprises third images; the third images are in a one-to-one correspondence with the second images; image content of the first area in each of the third images comes from the second images; and image content of areas, other than the first area, in the third images comes from the first image; and
generate a video stream based on the third image sequence and transmit the video stream to a user.
For claim 12, Examiner believes this claim should be amended in the following manner:
The electronic device according to claim 11, wherein the computer-executable instructions that cause the processor to generate [[a]] the third image sequence based on the first image sequence and the second image sequence further cause the processor to:
augment the first image sequence based on the second transmission frame rate, wherein [[the]] first images in the augmented first image sequence are in a one-to-one correspondence with the second images;
replace the image content of the first area in [[the]] each first image with [[the]] each corresponding second image, wherein the first images are the third images; and
generate the third image sequence based on the third images.
For claim 13, Examiner believes this claim should be amended in the following manner:
The electronic device according to claim 12, wherein the computer-executable instructions that cause the processor to replace the image content of the first area in [[the]] each first image with [[the]] each corresponding second image further cause the processor to:
determine a coordinate correspondence between pixel points in [[the]] each corresponding second image and pixel points in the first area in [[the]] each first image based on a width and height of [[the]] each first image, a width and height of the first area, a spherical angle range covered by the first area, and coordinates of a center point of a video picture of the user; and
replace the pixel points in the first area in [[the]] each first image with the pixel points in [[the]] each corresponding second image based on the coordinate correspondence.
For claim 14, Examiner believes this claim should be amended in the following manner:
The electronic device according to claim 11, wherein the second image sequence further comprises a fourth image; image content of the fourth image is [[the]] same as image content of a second area in the first image; and the second area is located in areas other than the first area; and the processor is further caused to:
determine whether to update the first image, based on a similarity between the image content of the fourth image and the image content in the second area in the first image; and
if so, update the first image.
For claim 15, Examiner believes this claim should be amended in the following manner:
The electronic device according to claim 11, wherein after the video stream is transmitted to the user, the processor is further caused to:
update coordinates of a center point of the first area in the first image based on [[the]] coordinates of [[the]] a center point of [[the]] a video picture of the user when watching the video stream; and
update the first area in the first image based on the updated coordinates of the center point of the first area.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3, 6 and 11-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu (U.S. Patent Application Publication 2016/0165233 A1) in view of Kim et al., Region of Interest-based Segmented Tiled Adaptive Streaming Using Head-mounted Display Tracking Sensing Data, International Journal of Distributed Sensor Networks, Vol. 15(12), November 2019 (hereinafter “Kim”).
For claim 1, Liu discloses an image processing method (disclosing a system and method for processing images (par. 1)), comprising: receiving a first image sequence transmitted based on a first transmission frame rate and a second image sequence transmitted based on a second transmission frame rate, wherein the first image sequence comprises a first image (disclosing the method receives a first image sequence for transmission at a first frame rate and a second image sequence for transmission at a second frame rate where the first image sequence comprises a first image (par. 204, 207 and 232)); the second image sequence comprises second images (disclosing the second image sequence comprises second images (par. 204)); image content of each of the second images is the same as image content of a first area in the first image (disclosing image content of each of the second images corresponds and is same as a region of interest of a first area in the first image (par. 204)); and the first transmission frame rate is less than the second transmission frame rate (disclosing the first frame rate for transmitting the first image sequence is less than the second frame rate for transmitting the second image sequence (par. 204)).
Liu does not disclose generating a third image sequence based on a first image sequence and a second image sequence, wherein the third image sequence comprises third images; the third images are in a one-to-one correspondence with second images; image content of a first area in each of the third images comes from the second images; and image content of areas, other than the first area, in the third image comes from a first image; and generating a video stream based on the third image sequence and transmitting the video stream to a user.
However, these limitations are well-known in the art as disclosed in Kim.
Kim similarly discloses a system and method for transmitting video as image sequences for streaming to a user (page 1). Kim likewise discloses LQ Tiles as a first image sequence and HQ Tiles as a second image sequence (pages 3-4 and 13-14). Kim discloses a third image sequence of third images where the third images correspond to second images in the second image sequence (pages 3-4 and 13-14). Kim further explains image content of a region of interest as a first area in the third images comes from the second images and image content of areas other than the region of interest comes from first images of the first image sequence so that a video stream is generated based on the third image sequence for transmission to a user for streaming (pages 1, 3-4 and 13-14). It follows Liu may be accordingly modified with the teachings of Kim to generate a third image sequence based on its first image sequence and its second image sequence to generate a video stream for transmission to a user.
A person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention would find it obvious to modify Liu with the teachings of Kim. Kim is analogous art in dealing with a system and method for transmitting video as image sequences for streaming to a user (page 1). Kim discloses its generation of a third image sequence is advantageous in generating an adaptive video stream for appropriate presentation of a region of interest to a user (pages 1, 3-4 and 13-14). Consequently, a PHOSITA would incorporate the teachings of Kim into Liu for generating an adaptive video stream for appropriate presentation of a region of interest to a user. Therefore, claim 1 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
For claim 2, depending on claim 1, Liu as modified by Kim discloses wherein the generating a third image sequence based on the first image sequence and the second image sequence comprises: augmenting the first image sequence based on the second transmission frame rate, wherein the first images in the augmented first image sequence are in a one-to-one correspondence with the second images; replacing image content of the first area in the first image with the corresponding second image, wherein the first images are the third images; and generating the third image sequence based on the third images (Liu discloses receives a first image sequence for transmission at a first frame rate and a second image sequence for transmission at a second frame rate where the first image sequence comprises a first image and the second image sequence comprises second images (par. 204); Kim similarly discloses a system and method for transmitting video as image sequences for streaming to a user (page 1). Kim likewise discloses LQ Tiles as a first image sequence and HQ Tiles as a second image sequence (pages 3-4 and 13-14); Kim explains the first image sequence corresponds to the second images of the second image sequence so that the first image sequence is augmented by replacing image content of a region of interest as a first area in each first image with each corresponding second image so that the augmented first images become third images of a third image sequence of a video stream for transmission to a user (pages 1, 3-4 and 13-14); and it follows Liu may be accordingly modified with the teachings of Kim to augment its first image sequence based on its second transmission frame rate with its second image sequence to generate its third image sequence to appropriately generate a video stream for transmission to a user).
For claim 3, depending on claim 2, Liu as modified by Kim discloses wherein the replacing image content of the first area in the first image with the corresponding second image comprises: determining a coordinate correspondence between pixel points in the second image and pixel points in the first area in the first image based on a width and height of the first image, a width and height of the first area, a spherical angle range covered by the first area, and coordinates of a center point of a video picture of the user; and replacing the pixel points in the first area in the first image with the pixel points in the corresponding second image based on the coordinate correspondence (Kim similarly discloses a system and method for transmitting video as image sequences for streaming to a user (page 1). Kim likewise discloses LQ Tiles as a first image sequence and HQ Tiles as a second image sequence (pages 3-4 and 13-14); Kim explains the first image sequence corresponds to the second images of the second image sequence so that the first image sequence is augmented by replacing image content of a region of interest as a first area in each first image with each corresponding second image so that the augmented first images become third images of a third image sequence of a video stream for transmission to a user (pages 1, 3-4 and 13-14); Kim further explains its system determines a coordinate correspondence between pixel points in each corresponding second image and pixel points in the region of interest in each first image based on a width and height of each first image, a width and height of the region of interest, a spherical angle range covered by the region of interest and coordinates of a center point of a video picture of the user to replace the pixel points in the region of interest in each first image with the pixel points in each corresponding second image based on the coordinate correspondence (pages 3-4, 6-7 and 13-14); and it follows Liu may be accordingly modified with the teachings of Kim to generate a third image sequence based on its first image sequence and its second image sequence to generate a video stream for transmission to a user).
For claim 6, Liu as modified by Kim discloses an image processing method (Liu discloses a system and method for processing images (par. 1)), comprising: transmitting a first image sequence based on a first transmission frame rate, and transmitting a second image sequence based on a second transmission frame rate, wherein the first image sequence comprises a first image (Liu discloses the method receives a first image sequence for transmission at a first frame rate and a second image sequence for transmission at a second frame rate where the first image sequence comprises a first image (par. 204, 207 and 232)); the second image sequence comprises second images (Liu discloses the second image sequence comprises second images (par. 204)); image content of each of the second images is the same as image content of a first area in the first image (Liu discloses image content of each of the second images corresponds and is same as a region of interest of a first area in the first image (par. 204)); and the first transmission frame rate is less than the second transmission frame rate (Liu discloses the first frame rate for transmitting the first image sequence is less than the second frame rate for transmitting the second image sequence (par. 204); Kim similarly discloses a system and method for transmitting video as image sequences for streaming to a user (page 1); Kim likewise discloses LQ Tiles as a first image sequence and HQ Tiles as a second image sequence (pages 3-4 and 13-14); Kim discloses a third image sequence of third images where the third images correspond to second images in the second image sequence (pages 3-4 and 13-14); Kim further explains image content of a region of interest as a first area in the third images comes from the second images and image content of areas other than the region of interest comes from first images of the first image sequence so that a video stream is generated based on the third image sequence for transmission to a user for streaming (pages 1, 3-4 and 13-14); and it follows Liu may be accordingly modified with the teachings of Kim to generate a third image sequence based on its first image sequence and its second image sequence to generate a video stream for transmission to a user).
For claim 11, Liu as modified by Kim discloses an electronic device (Liu discloses a computer as an electronic device (Figs. 3-4; par. 283)), comprising: a processor (Liu discloses a processor (par. 211)); and a memory configured to store computer-executable instructions (Liu discloses a memory for storing computer-executable instructions for execution by the processor (par. 211 and 283)) that, when executed, cause the processor to perform the method of claim 1 (see above as to claim 1).
For claim 12, depending on claim 11, this claim is a combination of the limitations of claim 11 and claim 2. It follows claim 12 is rejected for the same reasons as to claim 11 and claim 2.
For claim 13, depending on claim 12, this claim is a combination of the limitations of claim 12 and claim 3. It follows claim 13 is rejected for the same reasons as to claim 12 and claim 3.
Claim(s) 5, 10 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kim further in view of Doron (U.S. Patent 10,666,941 B1).
For claim 5, depending on claim 1, Liu as modified by Kim does not disclose updating coordinates of a center point of a region of interest in an image based on coordinates of a center point of a video picture of a user.
However, these limitations are well-known in the art as disclosed in Doron.
Doron similarly discloses a system and method for live streaming video to a user (col. 1/lines 12-15). Doron explains its system updates coordinates of a center point for a region of interest in an image based on coordinates of a center point of a video picture for a user (col. 11/lines 32-54). It follows Liu and Kim may be accordingly modified with the teachings of Doron to update coordinates of a center point of a region of interest in an image based on coordinates of a center point of a video picture of its user watching its video stream for updating its first area in its first image based on the updated coordinates of the center point of its first area.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Liu and Kim with the teachings of Doron. Doron is analogous art in dealing with a system and method for live streaming video to a user (col. 1/lines 12-15). Doron discloses its update for coordinates of a center is advantageous in appropriately adjusting image quality for video streaming (col. 11/lines 32-54). Consequently, a PHOSITA would incorporate the teachings of Doron into Liu and Kim for appropriately adjusting image quality for video streaming. Therefore, claim 5 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
For claim 10, depending on claim 6, Liu as modified by Kim and Doron discloses wherein after transmitting the first image sequence based on the first transmission frame rate, and transmitting the second image sequence based on the second transmission frame rate, the method further comprises: obtaining the updated coordinates of the center point of the first area; and updating the first area in the first image based on the updated coordinates of the center point of the first area (Doron similarly discloses a system and method for live streaming video to a user (col. 1/lines 12-15); Doron explains its system updates coordinates of a center point for a region of interest in an image based on coordinates of a center point of a video picture for a user (col. 11/lines 32-54); and it follows Liu and Kim may be accordingly modified with the teachings of Doron to update coordinates of a center point of a region of interest for updating its first area in its first image based on the updated coordinates of the center point of its first area).
For claim 15, depending on claim 11, this claim is a combination of the limitations of claim 11 and claim 5. It follows claim 15 is rejected for the same reasons as to claim 11 and claim 5.
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kim further in view of Rajvanshi et al. (U.S. Patent Application Publication 2019/0320113 A1, hereinafter “Rajvanshi”) and Bakunov et al. (U.S. Patent Application Publication 2024/0297957 A1, hereinafter “Bakunov”).
For claim 7, depending on claim 6, Liu as modified by Kim does not disclose capturing an image according to a preset capture frame rate.
However, these limitations are well-known in the art as disclosed in Rajvanshi.
Rajvanshi similarly discloses a system and method for generating a video stream for presentation to a user (par. 23). Rajvanshi explains images captured for providing the video stream may be captured as a preset capture frame rate such as 24 or 30 frames per second (par. 128). It follows Liu and Kim may be accordingly modified with the teachings of Rajvanshi to capture its first image according to a preset capture frame rate.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Liu and Kim with the teachings of Rajvanshi. Rajvanshi is analogous art in dealing with a system and method for generating a video stream for presentation to a user (par. 23). Rajvanshi discloses its use of a preset capture frame rate is advantageous in appropriately capturing image frames for providing a video stream to a user (par. 128). Consequently, a PHOSITA would incorporate the teachings of Rajvanshi into Liu and Kim for appropriately capturing image frames for providing a video stream to a user.
Liu as modified by Kim and Rajvanshi does not disclose determining a first area in an image and cropping the first area in the image to obtain a second image.
However, these limitations are well-known in the art as disclosed in Bakunov.
Bakunov similarly discloses a system and method for generating a video stream for presentation to a user (par. 41). Bakunov explains its system determines a region of interest as a first area in an image and crops the region of interest in the image to obtain a second image (par. 89 and 114-115). It follows Liu, Kim and Rajvanshi may be accordingly modified with the teachings of Bakunov to determine its first area in its first image and cropping its first area in its first image to obtain its second image.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Liu, Kim and Rajvanshi with the teachings of Bakunov. Bakunov is analogous art in dealing with a system and method for generating a video stream for presentation to a user (par. 41). Bakunov discloses its use of cropping is advantageous in appropriately obtaining a region of interest from an image for presentation to a user (par. 89 and 114-115). Consequently, a PHOSITA would incorporate the teachings of Bakunov into Liu, Kim and Rajvanshi for appropriately obtaining a region of interest from an image for presentation to a user. Therefore, claim 7 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Kim further in view of Rajvanshi and Bakunov further in view of Huang et al. (U.S. Patent Application Publication 2012/0002112 A1, hereinafter “Huang”).
For claim 8, depending on claim 7, Liu as modified by Kim, Rajvanshi and Bakunov does not disclose determining a first area in an image based on a difference in image content of a plurality of images with adjacent timestamps.
However, these limitations are well-known in the art as disclosed in Huang.
Huang similarly discloses a system and method for generating a video stream for presentation to a user (par. 51-52). Huang explains its system determines a region of interest as a first area in an image based on differences of image content of a plurality of adjacent images with adjacent timestamps (par. 61-64). It follows Liu, Kim, Rajvanshi and Bakunov may be accordingly modified with the teachings of Huang to determine its first area in its first image based on a difference in image content of a plurality of first images with adjacent timestamps.
A PHOSITA before the effective filing date of the claimed invention would find it obvious to modify Liu, Kim, Rajvanshi and Bakunov with the teachings of Huang. Huang is analogous art in dealing with a system and method for generating a video stream for presentation to a user (par. 51-52).Huang discloses its determination of differences between image content is advantageous in appropriately finding a region of interest in an image for processing and presentation (par. 61-64). Consequently, a PHOSITA would incorporate the teachings of Huang into Liu, Kim, Rajvanshi and Bakunov for appropriately finding a region of interest in an image for processing and presentation. Therefore, claim 8 is rendered obvious to a PHOSITA before the effective filing date of the claimed invention.
Allowable Subject Matter
Claims 4, 9 and 14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHARLES TSENG whose telephone number is (571)270-3857. The examiner can normally be reached 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHARLES TSENG/ Primary Examiner, Art Unit 2613
Prosecution Insights