DETAILED ACTIONNotice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 7/14/2026 has been entered.
Examiner's Note
The instant application has a lengthy prosecution history and the examiner encourages the applicant to have a telephonic interview with the examiner prior to filing a response to the instant office action. Also, prior to the interview the examiner encourages the applicant to present multiple possible claim amendments, so as to enable the examiner to identify claim amendments that will advance prosecution in a meaningful manner.
Acknowledgment
Claims 1, 12, and 16, amended on 7/14/2026, are acknowledged by the examiner.
Claims 3 and 14-15 were canceled. They are acknowledged by the examiner.
Response to Arguments
Presented arguments with respect to claims 1, 12, and 16, and their dependent claims have been fully considered, but some are rendered moot in view of the new ground of rejection necessitated by amendments initiated by the applicants. Examiner addresses the main arguments of the Applicant as below.
Claim Rejection – 35 U.S.C. § 112
The following is a quotation of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by the inventor of carrying out his invention.
The following is a quotation of 35 U.S.C. 112(b):
(B) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of pre-AIA 35 U.S.C. 112, second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-2, 4-13, and 16 are rejected under 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph because of a new matter. Amended claims 1, 12, and 16 include “wherein the second image is independent of the input sequence”. However, the application specification does not teach “wherein the second image is independent of the input sequence”. Hence, the claim limitation “wherein the second image is independent of the input sequence” is a new matter, which is not described in the application as originally filed. The new matter is required to be canceled from the claims (Please see MPEP 608.04).
Claims 1-2, 4-13 and 16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter, which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention. Amended claims 1, 12, and 16 include “to project the plurality of features onto a lower dimension in a feature space”. The meaning of this limitation is unclear. First, the application specification has never mentioned “a lower dimension”. Second, it is not clear “a lower dimension” of what component. Moreover, it is also not clear how many dimensions exist in the feature space. Therefore, these claims and their dependent claims are indefinite are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims under pre-AIA 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly owned at the time any inventions covered therein were made absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and invention dates of each claim that was not commonly owned at the time a later invention was made in order for the examiner to consider the applicability of pre-AIA 35 U.S.C. 103(c) and potential pre-AIA 35 U.S.C. 102(e), (f) or (g) prior art under pre-AIA 35 U.S.C. 103(a).
Claims 1-2, 4-8, 12-13, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Rosewarne (AU Patent Application Publication 2021202140 A1), (“Rosewarne”), in view of Yang et al. (US Patent 11,501,447 B2), (“Yang”), in view of Gosling et al. (US Patent 11,431,990 B2), (“Gosling”).
Regarding claim 1, Rosewarne meets the claim limitations as follow.
A method of decoding feature information (A method of decoding feature maps from encoded data) [Rosewarne: 00016, Abstract] of an image (an image or video frame) [Rosewarne: para. 0006] performed by a decoding apparatus (an apparatus
for decoding feature maps) [Rosewarne: para. 00017], the method comprising:obtaining at least one feature map (obtained from the decoded metadata 155, such that each feature map belongs to one group and one or more groups are indicated in the feature map groups 820) [Rosewarne: para. 000137] for a first image included in an input sequence (obtained from a previously decoded frame) [Rosewarne: para. 000114]; determining at least one feature transform matrix for the feature map (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122]: and inversely transforming (the residual coefficients 636 are passed through an inverse secondary transform module 644, operating in accordance with the secondary transform index 688 to produce intermediate inverse transform coefficients) [Rosewarne: para. 000121] a plurality of features included in the feature map ((The intermediate inverse transform coefficients 642 are inverse quantised by a dequantiser module 640 according to the quantisation parameter 692 to produce inverse transform coefficients) [Rosewarne: para. 000121]; (inverse quantisation for samples decoded from the encoded data to the feature maps, wherein, for at least a part of the samples decoded from the encoded data, one parameter is shared by a group of samples corresponding to a plurality of feature maps) [Rosewarne: para. 00018] – Rosewarne teaches that feature map data are processed by the inverse quantize and the inverse transformation modules) based on the determined feature transform matrix (The inverse primary transform module 648 applies DCT-2 transforms horizontally and vertically, constrained by the maximum available transform size as described with reference to the forward primary transform module 626. The types of inverse transform performed by the inverse secondary transform module 644 correspond with the types of forward transform performed by the forward secondary transform module 630. The types of inverse transform performed by the inverse primary transform module 648 correspond with the types of primary transform performed by the primary transform module 626) [Rosewarne: para. 000121], wherein the at least one feature transform matrix (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122] comprises a global feature transform matrix commonly applied to two or more features ((The 'channels' dimension indicates the number of concurrent 'feature maps' for a given tensor and the height and width dimensions indicate the size of the feature maps at the particular stage of the CNN. Channel count varies through a CNN according to the network architecture. Feature map size also varies, depending on subsampling occurring in specific network layers.) [Rosewarne: para. 0005, 00056]; (In an arrangement of the CNN backbone 310, the dimensionality of the tensors and thus the size of the resulting feature maps is selected to be aligned to the block size of the VVC standard. With generally rectangular video and a default CTU size of 128x128, the feature map widths and heights may be powers of two, e.g. the sizes for the three layers may be 128x64, 64x32, and 32x 16. Feature map sizes being powers of two results in greater alignment of the packed features with available block sizes in the VVC standard resulting from quadtree, binary, or ternary splits and reduced potential for coding artefacts in one feature map being caused by the contents of an adjacent feature map) [Rosewarne: para. 000197] – Rosewarne discloses that dimensions of the tensor with associated with the transform matrix are selected to be aligned with the block sizes of the VVC standard and resulting in the greater alignment of the packed features. It can be considered as a global matrix of the claim limitation),
wherein the global feature transform matrix is a predefined matrix (one matrix selected from a predefined set of matrices) [Rosewarne: para. 00109] to project the plurality of features (The group determiner module 510 assigns the feature maps (channels) of the input tensors 115 into feature map groups 512, based either on a predetermined criteria or on some measure of the data present in the tensors 115) [Rosewarne: para. 00087] onto a lower dimension in a feature space (an output tensor from the convolution has a lower spatial resolution than a corresponding input tensor. Operations such as 'max pooling' also reduce spatial size of
the output tensor compared to the input tensor) [Rosewarne: para. 00109], wherein the global feature transform matrix is generated based on a predetermined feature data set ((one matrix selected from a predefined set of matrices) [Rosewarne: para. 00109]; (The group determiner module 510 assigns the feature maps (channels) of the input tensors 115 into feature map groups 512, based either on a predetermined criteria or on some measure of the data present in the tensors 115) [Rosewarne: para. 00087]) obtained from a second image, and
wherein the second image is independent of the input sequence (one matrix selected from a predefined set of matrices) [Rosewarne: para. 00109].
Rosewarne does not explicitly disclose the following claim limitations (Emphasis added).
a predetermined feature data set obtained from a second image, and wherein the second image is independent of the input sequence.
However, in the same field of endeavor Yang further discloses the claim limitations and the deficient claim limitations as follows:
wherein the at least one feature transform matrix comprises a global feature transform matrix commonly applied to two or more features (applying the transform matrix to the object features resulting in transformed object features, and obtaining a predicted object mask associated with the second image frame by decoding the transformed object features.) [Yang: col. 2, line 7-11], and
a predetermined feature data set obtained from a second image (computing a transform matrix based on the image features of the first image frame and the image features of the second image) [Yang: col. 2, line 5-7].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne with Yang to program the system to implement of Yang’s method.
Therefore, the combination of Rosewarne with Yang will enable the system to improve feature coding efficiency [Yang: col. 6, line 55-67].
Rosewarne and Yang do not explicitly disclose the following claim limitations (Emphasis added).
wherein the second image is independent of the input sequence.
However, in the same field of endeavor Gosling further discloses the deficient claim limitations as follows:
wherein the second image is independent of the input sequence (The first and second images may form part of a video feed, with the position of the region of interest being updated for each image in the video feed in accordance with the attitude data for that image. Alternatively, the first and second images may be independent still images, with the first image forming a reference image in which the region of
interest may be defined by the user. The first and second images may be received at the same time, for instance, when the whole video is transferred in one go, or separately, as in the case of streaming video. The first image may be compressed
before the second image is received and/or compressed. Alternatively, the first and second images may be compressed in parallel) [Gosling: col. 5, line 49-61].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne and Yang with Gosling to program the system to implement of Gosling’s method.
Therefore, the combination of Rosewarne and Yang with Gosling will allow the fidelity of less important sections of an image to be reduced in order to provide an increased compression of the image, whilst also ensuring that the region of interest continues to encompass the most important section of the image even when the attitude of the camera has changed [Gosling: col. 3, line 18-22].
Regarding claims 2 and 13, Rosewarne meets the claim limitations as set forth in claims 1 and 12. Rosewarne further meets the claim limitations as follow.
wherein the number of global feature transform matrix is differently determined ((The backbone layers of the CNN may produce multiple tensors as output, for example, corresponding to different spatial scales of an input image represented by the video frame data 113. A 'feature pyramid network' (FPN) architecture may result in three tensors, corresponding to three layers, output from the backbone 114, with varying spatial resolution and channel count. The feature map quantiser and packer 116 receives tensors 115, which are output from the CNN backbone 114) [Rosewarne: para. 00087] – Note: Multiple tensors with varying spatial resolution indicate that the number of global feature transform matrix is differently determined) according to a type of a target task performed based on the feature map ((Fig. 1 is a schematic block diagram showing a distributed machine task system) [Rosewarne: 00024]; (The 'task performance' of a CNN is measured by comparing the result of the CNN in performing a task using specific input with a provided ground truth (i.e., 'training data'), generally prepared by humans and intended to indicate a 'correct' result) [Rosewarne: 0007] – Note: Paragraph [0290] of the application spec describes different tasks such as a machine task versus a hybrid of a machine task versus and a human task. As a result, Rosewarne discloses the claim limitation).
Regarding claim 4, Rosewarne meets the claim limitations as set forth in claim 1. Rosewarne further meets the claim limitations as follow.
wherein the feature data set comprises at least one feature selected from among a plurality of features obtained from the second image (feature maps are extracted from each frame according to a packing format to produce unpacked feature maps 812. The unpacked feature maps 812 include sample values as present in the decoded frame 14 7. Packing formats are described further with reference to Figs. 11-13. The sets of feature maps in the unpacked feature maps 812 are assigned to groups according to feature map groups 820, obtained from the decoded metadata 155, such that each feature map belongs to one group and one or more groups are indicated in the feature map groups 820) [Rosewarne: para. 000137; Fig. 8].
Rosewarne does not explicitly disclose the following claim limitations (Emphasis added).
a plurality of features obtained from a second image.
However, in the same field of endeavor Yang further discloses the claim limitations and the deficient claim limitations as follows:
a plurality of features obtained from the second image (obtaining a predicted object mask associated with the second image frame by decoding the transformed object features) [Yang: col. 2, line 7-11].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne with Yang to program the system to implement of Yang’s method.
Therefore, the combination of Rosewarne with Yang will enable the system to improve feature coding efficiency [Yang: col. 6, line 55-67].
Regarding claim 5, Rosewarne meets the claim limitations as set forth in claim 4. Rosewarne further meets the claim limitations as follow.
wherein the selected feature has a modified data structure within the feature data set (Sample-wise interleaving results in the higher structural detail of the four feature maps being shared by the same coding tree structure, with detail between the four feature maps varying from sample to sample. Accordingly, a common coding
tree structure and shared residual (except for local differences necessary to code the adjacent samples of different feature maps) is achieved, resulting in an increase in compression efficiency. Once all groups of size four have been packed into the monochrome frame 1202 for a given layer, the remaining feature maps, such as feature map 1214, are packed adjacently, based on grouping but not in an interleaved manner. The remaining feature maps may be assigned to groups of any size as their group composition does not affect the packing process, aside from the order of packing. For the next layer, groups of four, such as group 1220 are packed in a sample-wise interleaved manner followed by feature maps belonging to groups of other sizes, such as feature map 1224. For the final layer, groups of four, such as group 1230 are packed in a sample-wise interleaved manner followed by feature maps belonging to groups of other sizes, such as feature map 1234) [Rosewarne: para. 000149; Fig. 12] – Please see details of the modified data structure in Fig. 12).
In addition, in the same field of endeavor Yang further discloses the claim limitations and the deficient claim limitations as follows:
a modified data structure (Communication media may be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal) [Yang: col. 14, line 60-67].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne with Yang to program the system to implement of Yang’s method.
Therefore, the combination of Rosewarne with Yang will enable the system to improve feature coding efficiency [Yang: col. 6, line 55-67].
Regarding claim 6, Rosewarne meets the claim limitations as set forth in claim 1. Rosewarne further meets the claim limitations as follow.
wherein the feature data set comprises only a plurality of regions of interest (ROls) obtained from the second image (The RPN head module 1020 performs a convolution on the input tensors, producing an intermediate tensor which is fed into two subsequent sibling layers, one for classifications and one for bounding box, or 'region of interest' (ROI), regression, as classification and bounding boxes 1022. The classification and bounding boxes 1022 are passed to an NMS module 1024 which prunes out redundant bounding boxes by removing overlapping boxes with a lower score to produce pruned bounding boxes 1026. The bounding boxes 1026 are passed to a region of interest (ROI) pooler 1028. The ROI pooler 1028 produces fixed-size feature maps from various input size maps using max pooling operations, where a subsampling takes the maximum value in each group of input values to produce one output value in the output tensor. Input to the ROI pooler 1028 are the P2-P5 feature maps 1010, 1012, 1014, and 1016, and region of interest proposals 1026. Each proposal (ROI) from 1026 is associated with a portion of the feature maps (1010-1016) to produce a fixed-size map.) [Rosewarne: para. 000143-000144; Fig. 10]).
Rosewarne does not explicitly disclose the following claim limitations (Emphasis added).
obtained from a second image.
However, in the same field of endeavor Yang further discloses the deficient claim limitations as follows:
obtained from the second image (obtaining a predicted object mask associated with the second image frame by decoding the transformed object features) [Yang: col. 2, line 7-11].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne with Yang to program the system to implement of Yang’s method.
Therefore, the combination of Rosewarne with Yang will enable the system to improve feature coding efficiency [Yang: col. 6, line 55-67].
Regarding claim 7, Rosewarne meets the claim limitations as set forth in claim 1. Rosewarne further meets the claim limitations as follow.
wherein the feature data set is individually generated for ROI features and non-ROI features obtained from the second image (The RPN head module 1020 performs a convolution on the input tensors, producing an intermediate tensor which is fed into two subsequent sibling layers, one for classifications and one for bounding box, or 'region of interest' (ROI)) [Rosewarne: para. 000143; Fig. 10]– Note: Rosewarne discloses that his invention generates both ROI (i.e. one for bounding box, or 'region of interest' (ROI)) and non-ROI (i.e. one for classifications)).
Rosewarne does not explicitly disclose the following claim limitations (Emphasis added).
obtained from a second image.
However, in the same field of endeavor Yang further discloses the deficient claim limitations as follows:
obtained from the second image (obtaining a predicted object mask associated with the second image frame by decoding the transformed object features) [Yang: col. 2, line 7-11].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne with Yang to program the system to implement of Yang’s method.
Therefore, the combination of Rosewarne with Yang will enable the system to improve feature coding efficiency [Yang: col. 6, line 55-67].
Regarding claim 8, Rosewarne meets the claim limitations as set forth in claim 1. Rosewarne further meets the claim limitations as follow.
wherein the feature transform matrix is determined based on matrix index information obtained from a bitstream (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122].
Regarding claim 11, Rosewarne meets the claim limitations as set forth in claim 1. Rosewarne further meets the claim limitations as follow.
wherein the feature transform matrix is determined based on matrix index information obtained from a bitstream (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122].
Regarding claim 12, Rosewarne meets the claim limitations as follow.
A method of encoding feature information (A method of decoding feature maps from encoded data) [Rosewarne: 00016, Abstract] of an image (an image or video frame) [Rosewarne: para. 0006] performed by a encoding apparatus (an apparatus
for decoding feature maps) [Rosewarne: para. 00020], the method comprising:obtaining at least one feature map (obtained from the decoded metadata 155, such that each feature map belongs to one group and one or more groups are indicated in the feature map groups 820) [Rosewarne: para. 000137] for a first image (obtained from a previously decoded frame) [Rosewarne: para. 000114]; determining at least one feature transform matrix for the feature map (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122]: and transforming (The transform is applied) [Rosewarne: para. 00013] a plurality of features included in the feature map ((samples to be encoded into the encoded data to encode the feature maps, wherein, for at least a part of the samples to be encoded into the encoded data, one parameter is shared by a group of samples corresponding to a plurality of feature maps) [Rosewarne: para. 00019]) based on the determined feature transform matrix (The inverse primary transform module 648 applies DCT-2 transforms horizontally and vertically, constrained by the maximum available transform size as described with reference to the forward primary transform module 626. The types of inverse transform performed by the inverse secondary transform module 644 correspond with the types of forward transform performed by the forward secondary transform module 630. The types of inverse transform performed by the inverse primary transform module 648 correspond with the types of primary transform performed by the primary transform module 626) [Rosewarne: para. 000121], wherein the at least one feature transform matrix (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122] comprises a global feature transform matrix commonly applied to two or more features ((The 'channels' dimension indicates the number of concurrent 'feature maps' for a given tensor and the height and width dimensions indicate the size of the feature maps at the particular stage of the CNN. Channel count varies through a CNN according to the network architecture. Feature map size also varies, depending on subsampling occurring in specific network layers.) [Rosewarne: para. 0005, 00056]; (In an arrangement of the CNN backbone 310, the dimensionality of the tensors and thus the size of the resulting feature maps is selected to be aligned to the block size of the VVC standard. With generally rectangular video and a default CTU size of 128x128, the feature map widths and heights may be powers of two, e.g. the sizes for the three layers may be 128x64, 64x32, and 32x 16. Feature map sizes being powers of two results in greater alignment of the packed features with available block sizes in the VVC standard resulting from quadtree, binary, or ternary splits and reduced potential for coding artefacts in one feature map being caused by the contents of an adjacent feature map) [Rosewarne: para. 000197] – Rosewarne discloses that dimensions of the tensor with associated with the transform matrix are selected to be aligned with the block sizes of the VVC standard and resulting in the greater alignment of the packed features. It can be considered as a global matrix of the claim limitation), and wherein the global feature transform matrix is predefined by being generated in advance (one matrix selected from a predefined set of matrices) [Rosewarne: para. 00109] based on a predetermined feature data set (The group determiner module 510 assigns the feature maps (channels) of the input tensors 115 into feature map groups 512, based either on a predetermined criteria or on some measure of the data present in the tensors 115) [Rosewarne: para. 00087] obtained from a second image, and
wherein the global feature transform matrix is generated by applying a predetermined (one matrix selected from a predefined set of matrices) [Rosewarne: para. 00109] dimensionality reduction algorithm to the feature data set (Where a convolution stage has a 'stride' greater than one, an output tensor from the convolution has a lower spatial resolution than a corresponding input tensor. Operations such as 'max pooling' also reduce spatial size of the output tensor compared to the input tensor.) [Rosewarne: para. 0004].
Rosewarne does not explicitly disclose the following claim limitations (Emphasis added).
a predetermined feature data set obtained from a second image, and wherein the second image is independent of the input sequence.
However, in the same field of endeavor Yang further discloses the claim limitations and the deficient claim limitations as follows:
wherein the at least one feature transform matrix comprises a global feature transform matrix commonly applied to two or more features (applying the transform matrix to the object features resulting in transformed object features, and obtaining a predicted object mask associated with the second image frame by decoding the transformed object features.) [Yang: col. 2, line 7-11], and
a predetermined feature data set obtained from a second image (computing a transform matrix based on the image features of the first image frame and the image features of the second image) [Yang: col. 2, line 5-7].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne with Yang to program the system to implement of Yang’s method.
Therefore, the combination of Rosewarne with Yang will enable the system to improve feature coding efficiency [Yang: col. 6, line 55-67].
Rosewarne and Yang do not explicitly disclose the following claim limitations (Emphasis added).
wherein the second image is independent of the input sequence.
However, in the same field of endeavor Gosling further discloses the deficient claim limitations as follows:
wherein the second image is independent of the input sequence (The first and second images may form part of a video feed, with the position of the region of interest being updated for each image in the video feed in accordance with the attitude data for that image. Alternatively, the first and second images may be independent still images, with the first image forming a reference image in which the region of
interest may be defined by the user. The first and second images may be received at the same time, for instance, when the whole video is transferred in one go, or separately, as in the case of streaming video. The first image may be compressed
before the second image is received and/or compressed. Alternatively, the first and second images may be compressed in parallel) [Gosling: col. 5, line 49-61].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne and Yang with Gosling to program the system to implement of Gosling’s method.
Therefore, the combination of Rosewarne and Yang with Gosling will allow the fidelity of less important sections of an image to be reduced in order to provide an increased compression of the image, whilst also ensuring that the region of interest continues to encompass the most important section of the image even when the attitude of the camera has changed [Gosling: col. 3, line 18-22].
Regarding claim 16, Rosewarne meets the claim limitations as follow.
A method for transmitting data ((methods produce output) [Rosewarne: para. 00076]; (The bitstream 121 is supplied to the transmitter 122 for transmission over the communications channel 130) [Rosewarne: para. 0056]; (applications for transmission and storage of video data) [Rosewarne: para. 0002]) comprising feature information of an image (an image or video frame) [Rosewarne: para. 0006], the method comprising: generating a bitstream ((entropy encoding into a bitstream) [Rosewarne: para. 0042]; (encoding a 'residual' into the bitstream) [Rosewarne: para. 0014]), wherein the bitstream is generated based on obtaining at least one feature map ((a bitstream holding encoded packed feature maps and associated metadata) [Rosewarne: para. 0013]; (obtained from the decoded metadata 155, such that each feature map belongs to one group and one or more groups are indicated in the feature map groups 820) [Rosewarne: para. 000137] for a first image (obtained from a previously decoded frame) [Rosewarne: para. 000114], determining at least one feature transform matrix for the feature map (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122], and transforming (The transform is applied) [Rosewarne: para. 00013] a plurality of features included in the feature map ((samples to be encoded into the encoded data to encode the feature maps, wherein, for at least a part of the samples to be encoded into the encoded data, one parameter is shared by a group of samples corresponding to a plurality of feature maps) [Rosewarne: para. 00019]) based on the determined feature transform matrix (The inverse primary transform module 648 applies DCT-2 transforms horizontally and vertically, constrained by the maximum available transform size as described with reference to the forward primary transform module 626. The types of inverse transform performed by the inverse secondary transform module 644 correspond with the types of forward transform performed by the forward secondary transform module 630. The types of inverse transform performed by the inverse primary transform module 648 correspond with the types of primary transform performed by the primary transform module 626) [Rosewarne: para. 000121]; and transmitting the data comprising the bitstream (The bitstream 121 is supplied to the transmitter 122 for transmission over the communications channel 130) [Rosewarne: para. 0056], wherein the at least one feature transform matrix (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122] comprises a global feature transform matrix commonly applied to two or more features ((The 'channels' dimension indicates the number of concurrent 'feature maps' for a given tensor and the height and width dimensions indicate the size of the feature maps at the particular stage of the CNN. Channel count varies through a CNN according to the network architecture. Feature map size also varies, depending on subsampling occurring in specific network layers.) [Rosewarne: para. 0005, 00056]; (In an arrangement of the CNN backbone 310, the dimensionality of the tensors and thus the size of the resulting feature maps is selected to be aligned to the block size of the VVC standard. With generally rectangular video and a default CTU size of 128x128, the feature map widths and heights may be powers of two, e.g. the sizes for the three layers may be 128x64, 64x32, and 32x 16. Feature map sizes being powers of two results in greater alignment of the packed features with available block sizes in the VVC standard resulting from quadtree, binary, or ternary splits and reduced potential for coding artefacts in one feature map being caused by the contents of an adjacent feature map) [Rosewarne: para. 000197] – Rosewarne discloses that dimensions of the tensor with associated with the transform matrix are selected to be aligned with the block sizes of the VVC standard and resulting in the greater alignment of the packed features. It can be considered as a global matrix of the claim limitation), and wherein the global feature transform matrix is predefined by being generated in advance (one matrix selected from a predefined set of matrices) [Rosewarne: para. 00109] based on a predetermined feature data set (The group determiner module 510 assigns the feature maps (channels) of the input tensors 115 into feature map groups 512, based either on a predetermined criteria or on some measure of the data present in the tensors 115) [Rosewarne: para. 00087] obtained from a second image, and
wherein the global feature transform matrix is generated by applying a predetermined (one matrix selected from a predefined set of matrices) [Rosewarne: para. 00109] dimensionality reduction algorithm to the feature data set (Where a convolution stage has a 'stride' greater than one, an output tensor from the convolution has a lower spatial resolution than a corresponding input tensor. Operations such as 'max pooling' also reduce spatial size of the output tensor compared to the input tensor.) [Rosewarne: para. 0004].
Rosewarne does not explicitly disclose the following claim limitations (Emphasis added).
a predetermined feature data set obtained from a second image, and wherein the second image is independent of the input sequence.
However, in the same field of endeavor Yang further discloses the claim limitations and the deficient claim limitations as follows:
wherein the at least one feature transform matrix comprises a global feature transform matrix commonly applied to two or more features (applying the transform matrix to the object features resulting in transformed object features, and obtaining a predicted object mask associated with the second image frame by decoding the transformed object features.) [Yang: col. 2, line 7-11], and
a predetermined feature data set obtained from a second image (computing a transform matrix based on the image features of the first image frame and the image features of the second image) [Yang: col. 2, line 5-7].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne with Yang to program the system to implement of Yang’s method.
Therefore, the combination of Rosewarne with Yang will enable the system to improve feature coding efficiency [Yang: col. 6, line 55-67].
Rosewarne and Yang do not explicitly disclose the following claim limitations (Emphasis added).
wherein the second image is independent of the input sequence.
However, in the same field of endeavor Gosling further discloses the deficient claim limitations as follows:
wherein the second image is independent of the input sequence (The first and second images may form part of a video feed, with the position of the region of interest being updated for each image in the video feed in accordance with the attitude data for that image. Alternatively, the first and second images may be independent still images, with the first image forming a reference image in which the region of
interest may be defined by the user. The first and second images may be received at the same time, for instance, when the whole video is transferred in one go, or separately, as in the case of streaming video. The first image may be compressed
before the second image is received and/or compressed. Alternatively, the first and second images may be compressed in parallel) [Gosling: col. 5, line 49-61].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne and Yang with Gosling to program the system to implement of Gosling’s method.
Therefore, the combination of Rosewarne and Yang with Gosling will allow the fidelity of less important sections of an image to be reduced in order to provide an increased compression of the image, whilst also ensuring that the region of interest continues to encompass the most important section of the image even when the attitude of the camera has changed [Gosling: col. 3, line 18-22].
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Rosewarne (AU Patent Application Publication 2021202140 A1), (“Rosewarne”), in view of Yang et al. (US Patent 11,501,447 B2), (“Yang”), in view of Gosling et al. (US Patent 11,431,990 B2), (“Gosling”), in view of Afrouzi et al. (US Patent 11,153,503 B1), (“Afrouzi”).
Regarding claim 9, Rosewarne and Yang meet the claim limitations as set forth in claim 1. Rosewarne and Yang further meet the claim limitations as follow.
wherein features (A method of decoding feature maps from encoded data) [Rosewarne: 00016, Abstract] obtained from the second image (obtaining a predicted object mask associated with the second image frame by decoding the transformed object features) [Yang: col. 2, line 7-11] are clustered and included in the feature data set (feature maps are extracted from each frame according to a packing format to produce unpacked feature maps 812. The unpacked feature maps 812 include sample values as present in the decoded frame 14 7. Packing formats are described further with reference to Figs. 11-13. The sets of feature maps in the unpacked feature maps 812 are assigned to groups according to feature map groups 820, obtained from the decoded metadata 155, such that each feature map belongs to one group and one or more groups are indicated in the feature map groups 820) [Rosewarne: para. 000137; Figs. 8, 11-13] – Rosewarne discloses that the features are packed in the packing format).
Rosewarne, Yang and Gosling do not explicitly disclose the following claim limitations (Emphasis added).
wherein features are clustered.
However, in the same field of endeavor Afrouzi further the deficient claim limitations as follows:
wherein features are clustered (In some embodiments, the processor may employ unsupervised learning or clustering to organize unlabeled data into groups based on their similarities. Clustering may involve assigning data points to clusters wherein data points in the same cluster are as similar as possible. In some
embodiments, clusters may be identified using similarity measures, such as distance.) [Afrouzi: col. 47, line 54-60].
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne, Yang and Gosling with Afrouzi to program the system to implement of Afrouzi’s method.
Therefore, the combination of Rosewarne and Yang with Afrouzi will enable the system to improve accuracy of the map [Afrouzi: col. 33, line 4-7].
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Rosewarne (AU Patent Application Publication 2021202140 A1), (“Rosewarne”), in view of Yang et al. (US Patent 11,501,447 B2), (“Yang”), in view of Gosling et al. (US Patent 11,431,990 B2), (“Gosling”), in view of Rosewarne et al. (US Patent 12,309,389 B2), (“Rosewarne_389”).
Regarding claim 10, Rosewarne and Yang meet the claim limitations as set forth in claim 1. Rosewarne and Yang further meet the claim limitations as follow.
wherein the feature transform matrix is determined ((applying the transform matrix to the object features resulting in transformed object features, and obtaining a predicted object mask associated with the second image frame by decoding the transformed object features.) [Yang: col. 2, line 7-11]; (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122]) based on a most probable matrix (MPM) list for a current feature, andwherein the MPM list comprises a feature transform matrix (selected matrix signalled in the bitstream 120 using an index to identify which matrix of the set of matrices is to be used) [Rosewarne: para. 000122] for a feature reconstructed before the current feature as a MPM candidate (The decoded syntax elements are used to reconstruct parameters within the video decoder 144. Parameters include residual coefficients (represented by an arrow 724), a quantisation parameter 774, a secondary transform index 770, and mode selection information such as an intra prediction mode (represented by an arrow 758). The mode selection information also includes information) [Rosewarne: 000129]; (The reconstructed transform coefficients 7 40 are passed to an inverse primary transform module 744. The module 744 transforms the coefficients 740 from the frequency domain back to the spatial domain. The inverse primary transform module 744 applies inverse DCT-2 transforms horizontally and vertically, constrained by the maximum available transform size as described with reference to the forward primary transform module 626. The result of operation of the module 744 is a block ofresidual samples, represented by an arrow 748. The block of residual samples 748 is equal in size to the corresponding CB. The residual samples 748 are supplied to a summation module 750. At the summation module 750 the residual samples 748 are added to a decoded PB (represented as 752) to produce a block of reconstructed samples) [Rosewarne: para. 000131-000132; Fig. 7]).
Rosewarne, Yang and Gosling do not explicitly disclose the following claim limitations (Emphasis added).
based on a most probable matrix (MPM) list.
However, in the same field of endeavor Afrouzi further the deficient claim limitations as follows:
a most probable matrix (MPM) list ((Blocks of size 4x4 have 35 possible matrix intra prediction modes, otherwise blocks not exceeding 8x8 in size (i.e. 4x8, 8x4, and 8x8) have 19 possible matrix intra prediction modes. For other block sizes there are 11 possible matrix intra prediction modes. Typically, a list of 'most probable matrix modes' (MPMs) is generated in the video encoder 114.) [Rosewarne_389: col. 15, line 61-67]; (For 4x4 blocks, the 35 possible MW modes can be encoded using a 5- or 6-bit code, with 5 bits used for MW modes 0-28 and 6 bits used for MW modes 29-34. For 4x8, 8x4, and 8x8 blocks, the 19 possible MIP modes may be encoded using a 4- or 5-bit code, with 4 bits used for MW modes 0-12 and 5 bits used for MIP modes 13-18. For other sized blocks, the 11 possible MW modes may be encoded using a 3- or 4-bit code, with 3 bits used for MW modes 0-4 and 4 bits used for MIP modes 5-10) [Rosewarne_389: col. 16, line 43-51]).
It would have been obvious to one with an ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Rosewarne, Yang and Gosling with Rosewarne_389 to program the system to implement of Rosewarne_389’s method.
Therefore, the combination of Rosewarne, Yang and Gosling with Rosewarne_389 will enable the system to reduce the worst case memory bandwidth of the coefficient memory, reducing complexity and hardware cost without resulting in a proportionate degradation in coding efficiency [Rosewarne_389: col. 19, line 20-23].
Reference Notice
Additional prior arts, included in the Notice of Reference Cited, made of record and not relied upon is considered pertinent to applicant's disclosure.
Schmidt; Arthur W. (US Patent US-3720912-A) teaches most probable matrix.
Rosewarne, Christopher James (AU 2021202142 A1) discloses an apparatus that generates encoded data including encoded data of a feature map based on a neural network.
Reference Notice
Additional prior arts, included in the Notice of Reference Cited, made of record and not relied upon is considered pertinent to applicant's disclosure.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Philip Dang whose telephone number is (408) 918-7529. The examiner can normally be reached on Monday-Thursday between 8:30 am - 5:00 pm (PST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath Perungavoor can be reached on 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Philip P. Dang/ Primary Examiner, Art Unit 2488
Prosecution Insights