DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Yang in view of Zhang
Claims 1-3, 9, 11-13, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al.(USPubN 2023/0269496; hereinafter Yang) in view of Zhang et al.(USPubN 2024/0305782; hereinafter Zhang).
As per claim 1, Yang teaches a computer-implemented method performed on a mobile device(“An image processing method, a camera assembly and a mobile terminal” in Abs), the method comprising:
recording an input video of a scene, wherein the input video has a first format(“The processor 20 is electrically connected to the image sensor 10. The processor 20 is configured to obtain an RGB image after processing the Bayer image with a first processing algorithm and a demosaicing algorithm” in Para.[0038]);
converting the input video to a second format by performing, with an image signal processor of the mobile device, frontend processing and conversion from a Red Green Blue (RGB) color space to a YUV color space, wherein the input video in the second format has a smaller file size than the input video in the first format(“mobile phones and other electronic devices are often equipped with a camera to take pictures” in Para.[0037], “obtain a YUV image after processing the fused RGB image with a second processing algorithm” in Para.[0038], “The size of the image can be reduced by down scaler” in Para.[0128]);
Yang is silent about receiving a request from a user for an enhanced video; transmitting the input video in the second format to a server for cloud processing; and receiving the enhanced video from the server.
Zhang teaches receiving a request from a user for an enhanced video; transmitting the input video in the second format to a server for cloud processing; and receiving the enhanced video from the server(“interaction between a mobile phone and a cloud according to an embodiment of this application. A device side indicates a mobile phone side, and a cloud side indicates a cloud side. As shown in FIG. 4, in this embodiment of this application, a mobile phone may include at least a camera module and an encoding module. When a user uses the mobile phone to take a picture, the mobile phone may capture a RAW image through the camera module. Then, the mobile phone may encode, through the encoding module, the RAW image captured by the camera module to obtain an encoded bitstream corresponding to the RAW image, and upload the encoded bitstream corresponding to the RAW image to the cloud” in Para.[0108], “he cloud may include at least a decoding module, a RAW domain post-processing module, an ISP module, a YUV domain post-processing module, and a first-format encoder. The cloud may decode, through the decoding module, the encoded bitstream corresponding to the RAW image from the mobile phone, to obtain a reconstructed RAW image. The RAW domain post-processing module, the ISP module, and the YUV domain post-processing module may sequentially perform RAW domain image processing, ISP processing, and YUV domain image processing on the reconstructed RAW image, and the YUV domain post-processing module outputs a frame of YUV image. The first-format encoder may encode, in a first format, the YUV image output by the YUV domain post-processing module, to finally obtain an image (for example, a JPEG image) in the first format. Then, the cloud may send the image in the first format back to the mobile phone.” in Para.[0109], Fig. 4).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Yang with the above teachings of Zhang in order to improve user experience.
As per claim 2, Yang and Zhang teach all of limitation of claim 1.
Yang teaches wherein the frontend processing includes one or more actions selected from a group of linearization, black-level correction, digital gain, green channel imbalance correction, lens shading correction, white balance adjustment, highlight recovery, and combinations thereof(“The front end image processing module is configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm. The first processing algorithm includes at least one of: Camera Interface (CAMIF), Pedestal Correction, linearization, phase pixel compensation (Position dependent intra prediction combination (PDPC)), channel gain, bad pixel correction/bad cluster correction/Adaptive Bayer Filter (BPC/BCC/ABF), Black Level Correction (BLC), lens roll off, and white balance” in Para.[0124]).
As per claim 3, Yang and Zhang teach all of limitation of claim 1.
Yang teaches wherein conversion to the YUV color space includes one or more actions selected from a group of spatial denoising, demosaicing, applying a color correction matrix, and converting a RGB matrix to a YUV format matrix(“The second processing algorithms include at least one of: color correction, global tone mapping, gamma correction, color space transform, and down scaler.” in Para.[0124]).
As per claim 9, Yang and Zhang teach all of limitation of claim 1.
Yang teaches further comprising: while recording the input video, recording a preview video of the scene; and prior to receiving the enhanced video from the server, providing an option to view the preview video, wherein the preview video is associated with a lower quality than the enhanced video(“the image processing method works in a preview mode or a video mode, and the image pipeline includes a front-end image processing module (Image front end, IFE). The front end image processing module is configured to implement the first processing algorithm, the demosaicing algorithm, and the second processing algorithm. The first processing algorithm includes at least one of: Camera Interface (CAMIF), Pedestal Correction, linearization, phase pixel compensation (Position dependent intra prediction combination (PDPC)), channel gain, bad pixel correction/bad cluster correction/Adaptive Bayer Filter (BPC/BCC/ABF), Black Level Correction (BLC), lens roll off, and white balance. The second processing algorithms include at least one of: color correction, global tone mapping, gamma correction, color space transform, and down scaler.” in Para.[0124]).
As per claim 11, Yang teaches a non-transitory computer-readable medium with instructions stored thereon that, when executed by one or more computers, cause the one or more computers to perform operations, the operations comprising(Para.[0145]) and the other limitations in the claim 11 has been discussed in the rejection claim 1 and rejected under the same rationale.
As per claim 12, the limitations in the claim 12 has been discussed in the rejection claim 2 and rejected under the same rationale.
As per claim 13, the limitations in the claim 13 has been discussed in the rejection claim 3 and rejected under the same rationale.
As per claim 16, Yang teaches a system comprising: a processor; and a memory coupled to the processor, with instructions stored thereon that, when executed by the processor, cause the processor to perform operations comprising (Para.[0038] and [0145]) and the other limitations in the claim 16 has been discussed in the rejection claim 1 and rejected under the same rationale.
As per claim 17, the limitations in the claim 17 has been discussed in the rejection claim 2 and rejected under the same rationale.
As per claim 18, the limitations in the claim 18 has been discussed in the rejection claim 3 and rejected under the same rationale.
Yang in view of Zhang and Hong
Claims 4, 7, 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al.(USPubN 2023/0269496; hereinafter Yang) in view of Zhang et al.(USPubN 2024/0305782; hereinafter Zhang) further in view of Hong(USPubN 2009/0174812).
As per claim 4, Yang and Zhang teach all of limitation of claim 1.
Yang and Zhang is silent about wherein converting the input video to the second format further includes: performing quantization of the first format that encodes the input video in 12 bits to the second format that encodes the input video in 10 bits; and interpolating the input video in the first format by adding frames to increase a number of Frames Per Second (FPS) for the input video in the second format.
Hong teaches wherein converting the input video to the second format further includes: performing quantization of the first format that encodes the input video in 12 bits to the second format that encodes the input video in 10 bits; and interpolating the input video in the first format by adding frames to increase a number of Frames Per Second (FPS) for the input video in the second format(“an encoder for real-time video to be transmitted through a channel with limited capacity can lower the encoded bit rate by increasing the quantization step size to reduce the number of bits per encoded frame and/or by reducing the frame rate such as by discarding frames prior to encoding. Indeed, decreasing a typical input frame rate of 30-60 frames per second (fps) down to 10-15 fps may still provide tolerable video. Furthermore, a decoder can increase the displayed frame rate (up-conversion) of a received low-frame-rate bit stream by creating new frames in-between decoded frames. A higher display frame rate (e.g., up-conversion from 10-15 fps to 30 fps) makes the display more realistic. Decoders typically up-convert by interpolating (with or without motion compensation) the decoded frames to create new in-between frames. In addition, frame rate conversion is important for improving video quality” in Para.[0004]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Yang and Zhang with the above teachings of Hong in order to improve the imaging quality.
As per claim 7, Yang and Zhang teach all of limitation of claim 1.
Yang teaches further comprising: obtaining camera sensor data from a camera sensor of the mobile device in a Bayer image format; performing remosaicing of the camera sensor data; and performing binning of the camera sensor data(“ The image processing method includes: obtaining a Bayer image and a panchromatic image, obtaining an RGB image after processing the Bayer image with a first processing algorithm and a demosaicing algorithm” in Abs).
As per claim 14, the limitations in the claim 14 has been discussed in the rejection claim 4 and rejected under the same rationale.
As per claim 19, the limitations in the claim 19 has been discussed in the rejection claim 4 and rejected under the same rationale.
Yang in view of Zhang and Menachem
Claims 5, 6, 15 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al.(USPubN 2023/0269496; hereinafter Yang) in view of Zhang et al.(USPubN 2024/0305782; hereinafter Zhang) further in view of Menachem(USPubN 2018/0227502).
As per claim 5, Yang and Zhang teach all of limitation of claim 1.
Yang teaches wherein: the first format is a Bayer image format; obtaining the input video in the first format includes obtaining camera sensor data from a camera sensor of the mobile device; and converting the input video to the second format includes converting the camera sensor data in the Bayer image format to a YUV layout(Para.[0038]).
Yang and Zhang is silent about converting the input video to the second format includes converting the camera sensor data in the Bayer image format to a YUV420 layout.
Menachem teaches converting the input video to the second format includes converting the camera sensor data in the Bayer image format to a YUV420 layout(Para.[0038]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Yang and Zhang with the above teachings of Menachem in order to improve the imaging quality.
As per claim 6, Yang, Zhang and Menachem teach all of limitation of claim 5.
Yang teaches wherein converting the input video to the second format includes performing swizzling using Y-as-green, RGGB-quadrants, RGGB-tracks, or YUV conversions(Para.[0038]).
As per claim 15, the limitations in the claim 15 has been discussed in the rejection claim 5 and rejected under the same rationale.
As per claim 20, the limitations in the claim 20 has been discussed in the rejection claim 5 and rejected under the same rationale.
Yang in view of Zhang and Saenz
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Yang et al.(USPubN 2023/0269496; hereinafter Yang) in view of Zhang et al.(USPubN 2024/0305782; hereinafter Zhang) further in view of Saenz et al.(USPubN 2010/0225594; hereinafter Saenz).
As per claim 8, Yang and Zhang teach all of limitation of claim 1.
Yang is silent about further comprising: displaying playback of the enhanced video on the mobile device; receiving user selection indicative of a pause of the enhanced video; and displaying, in a user interface, an enhanced frame from the enhanced video, wherein the user interface includes an option to download the enhanced frame.
Zhang teaches further comprising: displaying playback of the enhanced video on the mobile device (Para.[0074).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Yang with the above teachings of Zhang in order to improve user experience.
Saenz teaches receiving user selection indicative of a pause of the enhanced video; and displaying, in a user interface, an enhanced frame from the enhanced video, wherein the user interface includes an option to download the enhanced frame (Para.[0030]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Yang and Zhang with the above teachings of Saenz in order to improve the imaging quality.
Yang in view of Zhang and Woodall
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Yang et al.(USPubN 2023/0269496; hereinafter Yang) in view of Zhang et al.(USPubN 2024/0305782; hereinafter Zhang) further in view of Woodall et al.(USPubN 2021/0258510; hereinafter Woodall)
As per claim 10, Yang and Zhang teach all of limitation of claim 9.
Yang teaches further comprising: performing, with the image signal processor, frontend processing of the preview video, conversion from the RGB color space to the YUV color space, demosaicing, applying a color correction matrix(Para.[0124]).
Yang and Zhang are silent about merging of long frames and short frames of the preview video to create merged frames.
Woodall teaches merging of long frames and short frames of the preview video to create merged frames (Para.[0067]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings Yang and Zhang with the above teachings of Woodall in order to improve the imaging quality.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SUNGHYOUN PARK whose telephone number is (571)270-1333. The examiner can normally be reached M - Thur 6:00 am - 4 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI Q TRAN can be reached at (571)272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SUNGHYOUN PARK/Examiner, Art Unit 2484
Prosecution Insights