3DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Yang et al. (US 20220284552 A1, hereinafter “Yang”)
Lin et al. (US 20240127411 A1, hereinafter “Lin”)
Pardeshi et al. (US 20210407051 A1, hereinafter ”Pardeshi”)
Response to Arguments
Applicant's arguments filed 01/18/2026 have been fully considered but they are not persuasive.
Applicant’s Arguments/Remarks (claims 1, 17, 29 and 30): applicant argues that the combination of the cited references does not teach or suggest "transmitting an indication of one or more masking parameters associated with an inpainting scheme" and "removing, according to the one or more masking parameters of the inpainting scheme, a portion of image data from an image to obtain a masked image, the masked image having a smaller amount of image data than the image".
Examiner’s Response. Examiner disagrees with applicant assertion above that th combination of the cited prior arts does not teach or suggest applicant invention as claimed and as argued above. Yang teaches “The flow validity mask may be a mask indicating which pixels in the optical flow are generally thought to be accurate and which pixels in the optical flow are generally thought to be inaccurate. A validity mask can be computed for both forward and backward flow due to the cycle structure of optical flow between two images. Once the optical flows and corresponding validity masks have been generated, the optical flows and corresponding validity masks are provided to the TSAM module… A TSAM module may be included in every bottleneck block in the encoding stage of the network and also included in the convolution layers in the decoding stage. The output of the encoder/decoder module may be an inpainted image having a resolution that is the same as the input resolution.”. in other words. the pixels and or the area to be masked is indicated and pass to the TSAM and the TSAM perform the inpainting by removing (Masking) those areas before transmission. Therefore, applicant arguments that the cited prior arts do not teach or suggest applicant invention as claimed has no merit.
Applicant arguments/Remarks (claims 11 and 27): applicant argues that Yang, Pardeshi, and Lin do not teach or suggest, "[transmitting] a preferred inpainting scheme to be applied to the masked image to the receiving device, wherein the inpainting scheme is based at least in part on the preferred inpainting scheme," as recited in dependent claim 11.
Examiner’s Response: Examiner disagrees with applicant assertion above that the combination of the cited prior arts does not teach or suggest applicant invention as claimed and as argued above. Yang teaches “In some examples, one or more CNN model parameters may also be received at the communication interface 204 and stored as the CNN model parameters 220. The CNN model parameters 220 may include one or more parameters and hyperparameters that define the CNN model. In examples, the CNN model parameters 220 may correspond to a specific CNN implementation that is to be implemented at the inpainting video server 202. For example, the CNN model parameters 220 may refer to a selection of a specific model (e.g., a specific model trained with a specific set of training data) that is made by a user (Yang [0035]). In other words, a preferred model for inpainting is selected and used accordingly. If understand applicant claim inpainting is performed at the transmitter is performed at the transmitter and an inpainted (image or video) is transmitted to the receiver. If the receiver is to performed the inversed of the inpainting that was done at the transmitter it would have been obvious to provide to the receiver the inpainting scheme to the receiver to facilitate the inverse process as it is well-known technique in the art.
Furthermore, applicant is reminded that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981).
Therefore, contrary to applicant assertion the combination of the cited prior arts teaches or suggest all of applicant invention as claimed. As a result, claims 1-30 are not allowable over the cited prior arts.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 17-19 and 29-30 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (US 20220284552 A1, hereinafter “Yang”) in view of Pardeshi et al. (US 20210407051 A1, hereinafter “Pardeshi”).
Regarding claims 1, 17, 29 and 30:
Yang teaches a user equipment (UE) for wireless communications (Yang [0001], [0030], fig. 1, where Yang teaches method and system with corresponding transmitter and receiver wherein in examples, the user 102 may utilize a computing device 104 to acquire and transmit the sequence of images 106 to the inpainting video server 122 via the network 118. The computing device 104, although depicted as a desktop computer for example, may be any one of a portable or non-portable computing device. For example, the computing device 104 may be a smartphone, a laptop, a desktop, a server), comprising:
one or more memories storing processor-executable code (Yang [0056], The method 600 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer readable medium, fig. 1); and one or more processors coupled with the one or more memories (Yang [0056], The method 600 can be executed as a set of computer-executable instructions executed by a computer system and encoded or stored on a computer readable medium, fig. 1); and individually or collectively operable to execute the code to cause the UE to:
transmit an indication of one or more masking parameters configured for inpainting scheme, the one or more masking parameters defining a portion of image data to remove from an image prior to image transmission (Yang [0027]-[0031], [0056], fig. 1, where Yang teaches in addition, a sequence of pixel mask images 110 may also be acquired and transmitted to the inpainting video server 122 via the network 118. Once the optical flows and corresponding validity masks have been generated, the optical flows and corresponding validity masks are provided to the TSAM module);
remove, by the UE and according to the one or more masking parameters of the inpainting scheme, the portion of image data from the image to obtain a masked image, the masked image having a smaller amount of image data than the image(Yang [0027]-[0030], [0056], fig. 1, where Yang teaches in examples, a user 102 may utilize the computing device 104 to generate one or more of the pixel mask regions 112. For example, a user may edit a video image to remove a watermark, object, or otherwise. In some examples, a video editing application may identify the pixel mask regions 112 for the sequence of pixel mask images 110 from an initial user selection. That is, a user may select a watermark in an image frame of video clip for removal; the video editing application or service may remove the watermark in all image frames of the video clip… A TSAM module may be included in every bottleneck block in the encoding stage of the network and also included in the convolution layers in the decoding stage. The output of the encoder/decoder module may be an inpainted image having a resolution that is the same as the input resolution.); and
transmit the compressed image to a receiving device (Yang [0030]-0031], [0034], [0056], fig. 1, where Yang teaches the inpainting video server 122 may then provide, or otherwise make available to the user, the completed sequence of images 126, where each image in the sequence of images 126 includes inpainted content. For example, an image 124 corresponding to image 114 may appear to be complete or otherwise include plausible content in regions previously having missing or corrupt pixel values, such as regions 116).
Yang fails to explicitly teach wherein compress, according to a compression scheme, the masked image to obtain a compressed image, the compressed image having a smaller amount of image data than the masked image based at least in part on the compression;
However, Yang teaches the sequence of images 106 may be acquired in any format and may be compressed and/or decompressed form. In addition, a sequence of pixel mask images 110 may also be acquired and transmitted to the inpainting video server 122 via the network 118). Pardeshi further teaches In at least one embodiment, ROP 1726 is a processing unit that performs raster operations such as stencil, z test, blending, and so forth. In at least one embodiment, ROP 1726 then outputs processed graphics data that is stored in graphics memory. In at least one embodiment, ROP 1726 includes compression logic to compress depth or color data that is written to memory and decompress depth or color data that is read from memory. In at least one embodiment, compression logic can be lossless compression logic that makes use of one or more of multiple compression algorithms. Compression logic that is performed by ROP 1726 can vary based on statistical characteristics of data to be compressed. For example, in at least one embodiment, delta color compression is performed on depth and color data on a per-tile basis (Pardeshi [0195]).
Therefore, taking the teachings of Yang and Pardeshi as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to obtain the image and compress or decompress it at needed according to a scheme and retransmit it according to any desired format such compressed or decompressed since performing compression and decompression are well known in the art and provide expected advantages when applied.
Regarding claims 2 and 18:
Yang fails to explicitly teach wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to: transmit an activation message initiating the inpainting scheme for the image to the receiving device, wherein the removing is based at least in part on the activation message.
However, Yang teaches in examples, the sequence of pixel mask images 110 and the sequence of images 106 may be provided to the inpainting video server 122. In examples, the communication interface 204 may be coupled to a network and receive the sequence of images 106 and sequence of pixel mask images 110; the sequence of images 106 may be stored as video frames 216 and the sequence of pixel mask images 110 may be stored as video frame masks 218 (Yang [0030-[0031], [0035]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to use any initialization sequence or flag as it is well known in the art or to use the transmission of the pixel mask image sequence as an activation for initiating the inpainting scheme for the image 106 received since the removing is based on the mask received and if there is no pixel mask image sequence no inpainting would be needed that way the sequence would serve dual purposes that would simplify the system.
Regarding claims 3 and 19:
Yang teaches wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to: receive an activation message from the receiving device initiating the inpainting scheme for the image, wherein the removing is based at least in part on the activation message (Yang [0030-[0031], [0035], see rejection of claim 2 above).
Claims 4-16, 20-28 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (US 20220284552 A1, hereinafter “Yang”) in view of Pardeshi et al. (US 20210407051 A1, hereinafter “Pardeshi”) and in view of Ln et al. (US 20240127411 A1, hereinafter “Lin”).
Regarding claims 4 and 20:
Yang in view of Pardeshi fails to teach wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to: transmit an image masking capability (interpreted as the making the UE is able to perfume) of the UE to the receiving device, wherein the inpainting scheme is based at least in part on the image masking capability.
However, Yang teaches where transmitting the pixel mask sequence is an indication that the system is capable of receiving and/or performing image masking (Yang [0030-[0031], [0035], see rejection of claim 2 above).
And Lin further teaches in one or more embodiments, the server(s) 104 includes all, or a portion of, the panoptic inpainting system 102. For example, the panoptic inpainting system 102 operates on the server(s) to generate and provide inpainted digital images. In some cases, the panoptic inpainting system 102 utilizes, locally on the server(s) 104 or from another network location (e.g., the database 112), a panoptic inpainting neural network 103 to generate inpainted digital images. In addition, the panoptic inpainting system 102 includes or communicates with a panoptic inpainting neural network 103 (and/or a semantic discriminator for training). In one or more embodiments, the client device 108 and the server(s) 104 work together to implement the panoptic inpainting system 102. For example, in some embodiments, the server(s) 104 train one or more neural networks discussed herein and provide the one or more neural networks to the client device 108 for implementation. In some embodiments, the server(s) 104 train one or more neural networks, the client device 108 request image edits, the server(s) 104 generate inpainted digital images and panoptic segmentation maps utilizing the one or more neural networks. Furthermore, in some implementations, the client device 108 assists in training one or more neural networks. (Lin 0032], [0049]-[0056]).
Therefore, taking the teaching of Yang, Pardeshi and Lin as whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application for the both the server and the client device to exchange information regarding of each other to perform inpainting, in order to decide what function or functions to run on the server or the client, in order to make the system more efficient.
Regarding claims 5 and 21:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more processors are individually or collectively further operable to execute the code to cause the VE to:
communicate with the receiving device to train the inpainting scheme to recover the image from the compressed image, the inpainting scheme based at least in part on the training (Yang [0053]-[0055], where Yang teaches the real-world application of video inpainting includes corrupted video restoration, object removal, watermark removal etc. To mimic these applications, the CNN encoder/decoder 406 of FIG. 4 may be trained and evaluated based on different types of masks, including but not limited to moving object-like masks, moving curve masks, and stationary masks. The object-like masks and the curve masks may include moving masks that occupy 0-10% to 60%-70% of the overall frame area and may move in both evaluation and training stages. The stationary mask is static in evaluation but may move during training for data augmentation).
Regarding claims 6 and 22:
Yang in view of Pardeshi fails to explicitly teach wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to:
receive a signal from the receiving device indicating one or more updated model parameters for the inpainting scheme, wherein the training is based at least in part on the one or more updated model parameters.
However, Yang teaches the system can select between a plurality of different model which includes different model parameters as well (Yang [0052]-[0055]).
Furthermore, Lin teaches the panoptic inpainting system can further determine various losses associated with the realism prediction, such as one or more adversarial losses associated with different discriminators, where the adversarial losses are used to adjust parameters of a neural network (e.g., the generator neural network and/or the semantic discriminator) to improve accuracy. Additional detail regarding the semantic discriminator and training neural networks using the semantic discriminator is provided below with reference to the figures. Lin further teaches in one or more embodiments, the server(s) 104 includes all, or a portion of, the panoptic inpainting system 102. For example, the panoptic inpainting system 102 operates on the server(s) to generate and provide inpainted digital images. In some cases, the panoptic inpainting system 102 utilizes, locally on the server(s) 104 or from another network location (e.g., the database 112), a panoptic inpainting neural network 103 to generate inpainted digital images. In addition, the panoptic inpainting system 102 includes or communicates with a panoptic inpainting neural network 103 (and/or a semantic discriminator for training). (Lin 0032], [0049]-[0056]).
Therefore, taking the teachings of Yang, Pardeshi and Lin as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to train the network at different location that has more powerful resources or more processing capability, such as a server, and provide the parameter to a location with less powerful resources or less processing capability such as a mobile phone, in order to optimize the system and make it more efficient.
Regarding claims 7 and 23:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more updated model parameters comprise at least one of one or more updated model weighting factors to be applied for the inpainting scheme, a subset of layers of the inpainting scheme to be trained during the training, an updated loss function to be applied for the inpainting scheme, one or more sensors associated with the training, or a combination thereof (Yang [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claims 8 and 24:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to:
transmit an updated model weighting factor to be applied for the inpainting scheme, wherein removing the portion of the image data is based at least in part on the updated model weighting factor (Yang [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claims 9 and 25:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to:
receive an updated model weighting factor to be applied for the inpainting scheme, wherein removing the portion of the image data is based at least in part on the updated model weighting factor (Yang [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claims 10 and 26:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to:
receive a set of available inpainting schemes associated with the receiving device; and Attorney
transmit a selected inpainting scheme from the set of available inpainting schemes to the receiving device, wherein the inpainting scheme comprises the selected inpainting scheme (Yang [0030], [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claims 11 and 27:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to:
transmit a preferred inpainting scheme to be applied to the masked image to the receiving device, wherein the inpainting scheme is based at least in part on the preferred inpainting scheme (Yang [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claims 12 and 28:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more processors are individually or collectively further operable to execute the code to cause the UE to:
receive a preferred inpainting scheme to be applied to the masked image from the receiving device, wherein the inpainting scheme is based at least in part on the preferred inpainting scheme (Yang [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claim 13:
Yang in view of Pardeshi and in view of Lin teaches wherein the portion of the image data is removed from the image or from a field-of-view region of the image, the field-of-view region based on one or more sensors associated with the UE (Yang [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claim 14:
Yang in view of Pardeshi and in view of Lin teaches wherein the one or more masking parameters comprise at least one of a masking region of the image, a masking shape, a masking size, a masking location within the image, a masking periodicity, a masking index from a set of masking indices associated with the UE, or a combination thereof (Yang [0030], [0052]-[0055]; Lin 0032], [0049]-[0056]).
Regarding claim 15:
Yang in view of Pardeshi and in view of Lin teaches wherein removing the portion of image data comprises at least one of removing a masked portion of the image, removing a set of masked portions from each image in a corresponding set of images, removing a frame image from a video, or a combination thereof (Yang [0030], [0052]-[0055], [0077]; Lin 0032], [0049]-[0056]).
Regarding claim 16:
Yang in view of Pardeshi and in view of Lin teaches wherein the compressed image is transmitted via a physical layer channel associated with a Uu interface, a PC5 interface, or both (Yang [0030], [0052]-[0055]; Lin 0032], [0049]-[0056]: the use of Uu and PC5 interface is known in the art for vehicle-to-everything (V2x) communication. Using the system in a vehicle such as for object detection in the road would have been obvious to one having ordinary skill in the art and to that Uu and PC5 would be the choice because it is common and when use provides predictable result).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 March 31, 2026