Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/10/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-6, 8-15 and 17-22 have been considered but are moot because they relate to newly amended claim limitations for which new art Hagai is provided.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 10 and 18 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 10 and 18 recite ‘decoding, by a decoder in a single pass, the first subset of tokens’. The applicant’s specification provides no description of decoding ‘in a single pass’. The closest disclosure appears to be par. 3 of the applicants specification which indicates that a segmentation system segments multiple objects ‘in a single pass’ by selectively decoding only object regions of a frame. However, the ‘single pass’ operation refers to the segmentation not the decoding. Thus the specification does not provide written description support for ‘single pass’ decoding.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8-15 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Schulze (20160150235) in view of Ogawa (2023/0154016) and in further view of Hagai et al (2002/0057705).
In regard to claim 1 Schulze discloses a method comprising:
receiving a frame depicting an object, the frame being one of a plurality of frames of a video sequence (Schulze Fig. 2 and pars 38-41 note receiving video from camera 210 further note Fig. 9A-B and pars 79-86 particularly note par. 80 a current frame including a moving object);
encoding a plurality of tokens of the frame, each token being a representation of a grid of pixels of a frame (Schulze Fig. 9A-B and pars 79-86 note foreground block 922 as a ‘token’ that represents a grid of foreground pixels);
determining first and second subset of tokens for decoding (Schulze pars 79-86 note dividing a block into foreground and background ‘tokens’ further note Fig. 4 and par. 44 a frame comprises a plurality of blocks, and hence a first subset of foreground ‘tokens’ and a second subset of background ‘tokens’ finally note Fig. 6A and pars 46-60 note a first subset of tokens, e.g. the foreground tokens, are selected to be encoded and decoded by an upper decoder as shown in Fig. 5 and 6A which shows layer L1 being decoded by the upper decoding process into quantized output Q1 residual block R1’ and decoded layer L1’)
decoding, by a decoder in a ‘single pass’, a first subset of tokens using a decoder (Schulze Fig. 6A and pars 59-60 note decoding the selected subset of tokens e.g. the foreground, into the decoded representation ‘L1’).
Schulze suggests that tokens may be divided into subsets based on moving and non-moving objects (Schulze par. 80 note foreground layer corresponds to a moving object). It is noted that Schulze does not disclose details of selecting subsets of tokens based on a confidence score. However, Ogawa discloses a method for selecting a subset of ‘tokens’ corresponding to a moving object based on a likelihood of a token satisfying a confidence threshold, wherein the confidence threshold based on a confidence score of the token including a past object in a past frame (Ogawa Fig. 5 and pars 48-52 note tracking the true motion of an object between frames, particularly note par. 52 note determining the position of the object in the past frame in the current frame based on a cross correlation score (confidence score) being above a threshold value). Ogawa further discloses that the subset of foreground ‘tokens’ includes first, second or more sets of tokens corresponding to plural foreground objects (Ogawa Figs. 7-8 and pars 132-136 note the detected dog and cat objects in Fig. 7A, also note the dog, bicycle and car objects detected in Fig. 8C also note par. 134 areas without detected objects are considered background).
It is therefore considered obvious that one of ordinary skill in the art before the effective filing date of the invention would recognize the advantage of incorporating a method of selecting ‘tokens’ corresponding to one or more moving objects as taught by Ogawa in the invention of Schulze in order to gain the advantage of obtaining accurate object motion as suggested by Ogawa (Ogawa par. 3 note high accuracy tracking).
Schulze and Ogawa teaches identifying a subset of tokens as a first, foreground, subset of tokens including one or more sets of tokens corresponding to separate objects, based on the likelihood of tokens satisfying a confidence threshold and identifying tokens as a second, ‘background’ subset of tokens based on the likelihood of the tokens not satisfying confidence thresholds as noted above. It is noted that Schulze and Ogawa does not disclose details of excluding a second, background, subset of tokens from the decoder. However, Hagai discloses prioritized video coding by selecting first and second subsets of image data corresponding to a background and foreground objects (Hagai Fig. 8 and pars 3-8 note composite image composed of a background 701 and two foreground objects A and B shown in 702 and 703 which are located in front of the background). Hagai further discloses excluding a second, background, subset of image data from the decoder (Hagai Figs. 1-3 and generally pars 80-111 note par. 81 coded signals Eg1-Egn corresponding to objects of the composite image, further note par. 89 ordering the encoded objects according to a priority, also note Fig. 3 and pars 91-99 determining whether each encoded object may be decoded in the time allotted and skipping decoding of objects which cannot be decoded in that time, finally note par. 104 priority may be determined according to foreground order hence the background will be the lowest priority encoded object will excluded form decoding).
It is therefore considered obvious that one of ordinary skill in the art before the effective filing date of the invention, would recognize the advantage of incorporating prioritized coding as taught by Hagai in the layered coding of Shultz and Ogawa in order to gain the advantage of adapting the video coding to the capabilities of the decoding apparatus as suggested by Hagai (par. 21).
In regard to claim 2 refer to the statements made in the rejection of claim 1 above. Schulze further discloses:
encoding a representation of visual information in the past frame (Schulze par. 80 note previously encoded frame) ;
creating an affinity matrix by comparing each token of the plurality of tokens to the encoded representation of visual information in the past frame (Schulze pars 79-86 note motion vectors indicating the location of the same pixels in a previous frame);
encoding a mask probability of a particular past object in the past frame (Schulze par. 82 note generating masks M1 and M2 based on the motion vector information); and
obtaining a memory readout for the particular past object by applying the encoded mask probability of the particular past object in the past frame to the affinity matrix (Schulze pars 79-80 note motion vectors represent predictions from pixels in past frames, further note Figs 5-6A and pars. 46-60 note predictions are obtained from the past frame (memory) to predict the pixels of the current block)
In regard to claim 3 refer to the statements made in the rejection of claim 2 above.
encoding an existence of each particular masked object of a plurality of masked objects (Schulze pars 46-60 and Fig. 5 note encoding each of a plurality of masked layers, any number of layers may be used further note pars 79-86 and Fig. 9A one or more masked layers may correspond to a moving object).
In regard to claim 4 refer to the statements made in the rejection of claim 3 above. Schulze and Ogawa further disclose:
determining the confidence score of the token including the past object in the past frame by applying the existence of the particular masked object to the memory readout for the particular past object (Ogawa pars 48-52 note determining a cross correlation between the identified object features of a past frame and detected features of a current frame).
In regard to claim 5 refer to the statements made in the rejection of claim 1 above. Schulze further discloses that the decoder is a set-based encoder (Schulze par. 44 note a frame is encoded as a set of blocks) and further comprising:
indexing the first subset of tokens (Schulze pars 66-67 note blocks are indexed by position x and y, note all blocks including the foreground blocks include position information) ; and
decoding the indexed first subset of tokens (Schulze Fig. 6A note decoding image blocks);
In regard to claim 6 refer to the statements made in the rejection of claim 1 above. Schulze further discloses that the decoder is a convolutional decoder (Schulze pars 61-63 note either selecting or merging layers for ‘convolutional decoder’) and further comprising:
masking each of the plurality of tokens of the frame that are not included in the first subset of tokens (Schulze Fig. 9A and pars 79-86 note masking pixels which are not included in the selected layer e.g. the background pixels are masked in the foreground layer).
In regard to claim 8 refer to the statements made in the rejection of claim 1 above. Schulze and Ogawa further discloses:
determining that the first set of tokens representing a grid of pixels in the frame is a number of pixels apart from the second set of tokens representing another grid of pixels in the frame (Ogawa Fig. 8 note distance between the detected dog object and the detected car object);
masking each of the plurality of tokens of the frame that are not included in the first subset of tokens (Schulze Figs. 8A, 9A and 10A note masks M1 and M2 are applied to all pixels or ‘tokens’ which do not apply to the current layer, further note pars 46-47 more than two layers may be generated and thus the masks M1 and M2 would ask any ‘tokens’ corresponding to a third or more layers);
combining the first set of tokens of the first subset and the second set of tokens of the third first subset into a single encoded representation (Schulze Fig. 5 and pars 46-60 note combining all ‘token’ layers into a single encoded representation 570, ); and
decoding the first subset into a single encoded representation (Schulze Fig. 6A and pars 46-60 note decoding the encoded representation 570).
In regard to claim 9 refer to the statements made in the rejection of claim 1 above. Schulze and Ogawa further discloses that the first subset of tokens comprises a active regions corresponding to the plurality of object of the frame (Schulze par. 80 note the foreground ‘tokens’ correspond to a moving object which represents an active region of the frame as opposed to a stationary object, further note Ogawa Figs 7-8 detecting a plurality of foreground objects).
Claims 10-15 and 17-20 describe an apparatus and a non-transitory computer readable media that cause a processor and a memory to perform steps that correspond to the method described in claims 1-6 and 8-9. Refer to the statements made in regard to claims 1-6 and 8-9 above for the rejection of claims 10-15 and 17-20 which will not be repeated here for brevity. In particular regard to claims 10 and 18 Schulze further discloses an apparatus comprising a processor and a memory that may receive instructions from a computer readable medium (Schulz Fig. 2 and pars 38-41 note processor 220 and memory 230, the memory 230 including non-transitory media storing instructions to be executed by the processor).
Allowable Subject Matter
Claims 21-22 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claims 21 and 22, in addition to the limitations of claims 1 and 10 from which they depend, require determining that the decoder is a convolutional decoder or a transformer based decoder and either merging or not merging the second set of tokens based on the determination.
The closest arts are Schulze, Ogawa and Hagai which disclose segmenting video into foreground and background objects, dividing the objects into prioritized layers and decoding a portion of the video objects based on decoder capabilities. However, none of the prior art teach merging or not merging the object layers based on whether the decoder is a convolutional or transform decoder as required by claims 21 and 22.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20030235338 A1 Dye, Thomas A.
US 20060013495 A1 Duan; Ling Yu et al.
US 20060245655 A1 Suino; Tooru et al.
US 20220182625 A1 DANIELSSON; Niclas et al.
US 7630570 B1 Haskell; Barin Geoffry et al.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMIAH CHARLES HALLENBECK-HUBER whose telephone number is (571)272-5248. The examiner can normally be reached Monday to Friday from 9 A.M. to 5 P.M.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Vaughn can be reached at (571)272-3922. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEREMIAH C HALLENBECK-HUBER/Primary Examiner, Art Unit 2481
Prosecution Insights