DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments and amendments in the Amendment filed July 17, 2026 (herein “Amendment”) regarding the rejection of claims 1–8 under 35 U.S.C. 103 have been fully considered but they are not persuasive. Upon further search and consideration of secondary reference Lin, given the broadest reasonable interpretation of the newly recited weighted input tokens (first and second) being “associated with different features of the one or more first features,” Lin teaches in cited fig. 3 not only that the Q, K and V values are weighted, but also that the input sequence to the graphormer encoder IS a sequence of input tokens some of which come from grid features, and others from global features, thus being associated with different features (grid versus global). Therefore, Lin is found to apply to the newly recited limitations of claim 1, in newly cited portions of Lin, where the rejection rationale has been updated below. Accordingly, while Applicant’s arguments and amendments have been fully considered, they are not persuasive and the rejection in view of Fu and Lin is maintained herein.
Applicant’s arguments and amendments in the Amendment with respect to the rejection of claims 9–10, 12–18 and 20 under 35 U.S.C. 103 have been fully considered and are persuasive. The rejection of claims 9–10, 12–18 and 20 under 35 U.S.C. 103 has been withdrawn, and these claims are allowed.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–3, and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Fu et al., “POS-BERT: Point Cloud One-Stage BERT Pre-Training,” arXiv:2204.00989v1 [cs.CV], https://doi.org/10.48550/arXiv.2204.00989 (herein “Fu”) in view of Lin et al., "Mesh Graphormer," 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 12919-12928, doi: 10.1109/ICCV48922.2021.01270 (herein “Lin”).
Regarding claim 1, with deficiencies of Fu noted in square brackets [], Fu teaches a method of estimating an [interaction with a device], the method comprising (Fu page 1, Introduction, learning point cloud representation promotes applications such as augmented reality):
configuring a first token and a second token of an estimation model according to one or more first features of a 3-dimensional (3D) object (Fu pages 4–5, fig. 2, section 3, POS-BERT model (estimation model) taking as input a raw point cloud, corresponding to a 3D data representation (see page 1, Introduction), is split into two point clouds, Pg and Pl, which respectively are processed by a PGE module and thereafter with a transformer based encoder to embed (configure) patches from the Pg, and patches from the Pl into respective patch tokens (first and second tokens));
applying a first weight to the first token to produce a first-weighted input token and applying a second weight that is different from the first weight to the second token to produce a second-weighted input token (Fu page 6, section 3.3, each encoder respective to the first and second patch tokens has a weight that it applies in the encoding according to equation (1), where θm is the weight of the momentum encoder, and θe is the weight of the encoder, thus different weights);
generating, by a first encoder layer of an estimation-model encoder of the estimation model, an output token based on [receiving] the first-weighted input token and the second-weighted input token [as inputs to the first encoder layer and calculating the output token from the first-weighted input token and the second-weighted input token, the first-weighted input token and the second-weighted input token being associated with different features of the one or more first features] (Fu page 6, section 3.4, the Momentum encoder and the Encoder respectively output values
O
m
i
and
O
e
i
from the weighted first and second input patch tokens, where page 4, section 3 teaches that a modeling loss is based on the Encoder outputs’ patch tokens and the Momentum Encoder outputs’ patch tokens, thus the output of the Momentum encoder and Encoder being tokens);
[generating, by the estimation model, an estimated output based on the output token]; and
performing an operation based on the estimated output (Fu pages 9–10, section 5.2, various “downstream tasks” which as tasks (operations) performed based on the estimated output, including 3D object classification).
While Fu teaches that its point-cloud pre-training method is useful for augmented reality applications, and therefore would be a method for augmented reality which would be obvious to a person having ordinary skill in the art (herein “PHOSITA”) to include interpreting user gestures as interactions with an augmented reality app or headset with predictable results, nonetheless, Fu does not explicitly teach/anticipate that the augmented reality includes interactions with an augmented reality device.
Lin teaches interaction with a device (Lin page 12919, Introduction, 3D human pose and mesh reconstruction for applying to human-computer interactions).
Lin further teaches receiving weighted input tokens as inputs to the first encoder layer and calculating the output token from the first-weighted input token and the second-weighted input token (Lin page 12921, fig. 3, section 3.1, input tokens into the multi-layer graphormer encoder as a whole, thus the first layer of that encoder, where input tokens weighted by Q query values, K keys values and V vertex values into a multi-layer graphormer encoder that outputs “Coarse Mesh Output Tokens” from these input Q, K, V weighted tokens), the first-weighted input token and the second-weighted input token being associated with different features of the one or more first features (Lin pages 12921–12922, fig. 3, section 4, the input tokens to the Multi-layer graphormer encoder consisting of two different feature types: grid features with fine-grained local details, and global features that do not have fine-grained local details).
Lin still further teaches generating, by the estimation model, an estimated output based on the output token (Lin pages 12921–12922, fig. 3, coarse mesh output tokens output from the multi-layer graphormer encoder are then upsampled using MLP to produce the estimate output full mesh).
Therefore, taking the teachings of Fu and Lin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the feature processing and mesh producing disclosed in Lin at least because doing so would improve spatial locality in the features (See Lin page 12921, right column), as well as avoiding redundancies to make training more efficient (See Lin page 12922, right column, ll. 4-5).
Regarding claim 2, Fu teaches further comprising: receiving, at a backbone of the estimation model, input data corresponding to the interaction with the device (Fu page 5, standard transformer used as encoder backbone, shown in fig. 2 as receiving local point cloud set data, where page 1, section 1 teaches point clouds as being 3D data representation for example with augmented reality and thus would be data corresponding to interaction with an augmented reality device). Fu does not teach the remainder of the limitations of claim 2.
Lin teaches extracting, by the backbone, the one or more first features from the input data (Lin pages 12921–12922, fig. 3, image of size 224x224 used as input and image grid features are extracted from the last convolution block in the CNN (convolutional neural network which is pre-trained in other task of feature extraction));
receiving, at a two-dimensional (2D) feature extraction model, the one or more first features from the backbone (Lin pages 12921–12922, fig. 3, a pooling and MLP operation received as input the grid features shown to be 2D);
extracting, by the 2D feature extraction model, one or more second features associated with the one or more first features, the one or more second features comprising one or more 2D features (Lin pages 12921–12922, fig. 3, the Pooling + MLP outputs a global feature vector from the grid features);
receiving, at the estimation-model encoder, data generated based on the one or more 2D features (Lin pages 12921–12922, fig. 3, multi-layer graphormer encoder receiving the grid features and the global feature vector); and
generating, by the estimation model, the estimated output based on the data generated based on the one or more 2D features (Lin pages 12921–12922, fig. 3, coarse mesh output tokens output from the multi-layer graphormer encoder are then upsampled using MLP to produce the estimate output full mesh).
Therefore, taking the teachings of Fu and Lin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the feature processing and mesh producing disclosed in Lin at least because doing so would improve spatial locality in the features (See Lin page 12921, right column), as well as avoiding redundancies to make training more efficient (See Lin page 12922, right column, ll. 4-5).
Regarding claims 3, with claim 3 as exemplary, Fu does not explicitly teach, but Lin teaches wherein the data generated based on the one or more 2D features comprises an attention mask (Lin page 12921, fig. 3, input tokens generated by multi-head self-attention include masked tokens denoted in fig. 3 as [MASK]).
Therefore, taking the teachings of Fu and Lin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the feature processing and mask from the multi-head self attention disclosed in Lin at least because doing so would improve spatial locality in the features (See Lin page 12921, right column), as well as avoiding redundancies to make training more efficient (See Lin page 12922, right column, ll. 4-5).
Regarding claim 8, Fu does not explicitly teach the limitations of claim 8. Lin teaches generating a 3D scene including a visual representation of the 3D object; and updating the visual representation of the 3D object based on the output token (Lin fig. 3, pages 12921–12922, a coarse mesh is generated, including a visual representation of the human subject shown in the input image, and this coarse mesh is refined into an output mesh, also a representation of the 3D object human subject based on the full mesh output tokens).
Therefore, taking the teachings of Fu and Lin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the coarse and full mesh processing disclosed in Lin at least because doing so would provide for combining self attentions and graph convolutions in a transformer for human mesh reconstruction that outperforms existing graph convolution networks (See Lin page 12919, fig. 1 caption).
Claims 4–5 are rejected under 35 U.S.C. 103 as being unpatentable over Fu and Lin as set forth above regarding claim 1, further in view of Yang et al., "Dynamic Iterative Refinement for Efficient 3D Hand Pose Estimation," 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, January 8, 2022, pp. 2703-2713, doi: 10.1109/WACV51458.2022.00276 (herein “Yang”), further in view of Goyal et al., “PoWER-BERT: Accelerating BERT Inference via Progressive Word-vector Elimination,” arXiv:2001.08950v5 [cs.LG], Sept. 8, 2020, https://doi.org/10.48550/arXiv.2001.08950 (herein “Goyal”).
Regarding claim 4, with deficiencies of Fu noted in square brackets [], Fu teaches wherein the first encoder layer of the estimation-model encoder [corresponds to a first BERT encoder of the estimation-model encoder] (Fu fig. 2, pages 5–6, momentum encoder), and the method further comprises: concatenating a token, associated with an output of the first [BERT] encoder, with at least one of [camera intrinsic-parameter data, three-dimensional (3D) hand-wrist data, or bone-length data] to generate concatenated data (Fu fig. 2, pages 4–5, patch tokens output from top processing branch including PGE model processing, are concatenated along the patch dimension with class tokens to get the transformer input for the transformer encoder); and
receiving the concatenated data at a second BERT encoder (Fu fig. 2, page 5, the concatenated tokens are input to the transformer encoder (BERT encoder)).
Fu does not explicitly teach corresponds to a first BERT encoder of the estimation-model encoder, the first BERT encoder, or camera intrinsic-parameter data, three-dimensional (3D) hand-wrist data, or bone-length data.
Yang teaches camera intrinsic-parameter data, three-dimensional (3D) hand-wrist data, or bone-length data (Yang page 2709, STB dataset including 3D annotations of 21 joints of a hand/wrist).
Goyal teaches corresponds to a first BERT encoder of the estimation-model encoder, the first BERT encoder (Goyal fig. 1, page 2, PoWERT-BERT architecture including a chain of 12 encoders in series, thus the output of one BERT encoder being input to the next BERT encoder in the chain).
Therefore, taking the teachings of Fu and Yang together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the hand/wrist data disclosed in Yang at least because doing so would provide for hand pose estimation approaches with improved accuracy and efficiency (See Yang Abstract).
Further, taking the teachings of Fu as modified by Yang and Goyal together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the BERT encoder chain disclosed in Goyal at least because doing so would provide for improving the inference time of a BERT system while maintaining the accuracy (See Goyal Abstract).
Regarding claim 5, with deficiencies of Fu noted in square brackets, Fu teaches [the first BERT encoder] and the second BERT encoder (Fu fig. 2, page 5 the transformer encoder (BERT encoder)) [are included in a chain of BERT encoders, the first BERT encoder and the second BERT encoder being separated by at least three BERT encoders of the chain of BERT encoders; and the chain of BERT encoders] comprises at least one BERT encoder having more than four encoder layers (Fu fig. 2, page 5, within the transformer encoder architecture is an MLP layer, a maxpool layer, and the transformer encoder layer, and multiple MPL layers in the projector, this at least 5 layers in the encoder).
Goyal teaches the first BERT encoder, are included in a chain of BERT encoders, the first BERT encoder and the second BERT encoder being separated by at least three BERT encoders of the chain of BERT encoders; and the chain of BERT encoders (Goyal fig. 1, page 2, considering encoder 1 as the first BERT encoder, within a chain of 12 encoders, including a second BERT encoder–encoder 5, between encoder 1 and encoder 5 are three BERT encoders, encoders 2, 3, and 4).
Therefore, taking the teachings of Fu as modified by Yang and Goyal together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the BERT encoder chain disclosed in Goyal at least because doing so would provide for improving the inference time of a BERT system while maintaining the accuracy (See Goyal Abstract).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Fu and Lin as set forth above regarding claim 1, further in view of Shen et al., “Patient-specific reconstruction of volumetric computed tomography images from a single projection view via deep learning. Nat Biomed Eng., Nov. 2019, 3(11):880-888. doi: 10.1038/s41551-019-0466-4 (herein “Shen”).
Regarding claim 6, with deficiencies of Fu noted in square brackets [], Fu teaches [a data set used to train the estimation model is generated based on two-dimensional (2D) image rotation and rescaling that is projected to three dimensions (3D) in an augmentation process;] and a backbone of the estimation model is trained using two optimizers (Fu fig. 2, pages 6–7, two loss functions are used to train the POS-BERT model, one for mask patch modeling loss (MPM) given in equation 2, and one for the global point clouds (GFC) given in equation 4, where the overall loss function depends on both the MPM and GFC loss functions).
Fu does not explicitly teach, but Shen teaches a data set used to train the estimation model is generated based on two-dimensional (2D) image rotation and rescaling that is projected to three dimensions (3D) in an augmentation process (Shen pages 10 and 5–6, augmented training datasets with rotational transformations, the datasets being of 2D projection images reshaping and resizing the images which are scalable to full-size images (rescaling)).
Therefore, taking the teachings of Fu and Shen together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the point cloud data processing of Fu with the data augmentation disclosed in Shen at least because doing so would provide for robustness in the training of a deep neural network (See Shen page 5).
Allowable Subject Matter
Claim 7 remains objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, for the same reasoning provided on pages 24–25 of the Non-Final Office Action issued August 26, 2025.
Independent claims 9 and 17, and therefore claims 10, 12–18 and 20 which depend therefrom are allowed. The closest cited prior art includes Fu, in view of Lin and Zhai, as applied in the Non-Final Office Action issued April 17, 2026. In particular, while Zhai teaches updating an attention mask regarding suppressing or ignoring occluded areas in an image, Zhai does not explicitly teach removing a portion of the attention mask associated with the occluded portion, as presently claimed via the newly amended limitations. In this way, Applicant’s arguments on page 16 of the Amendment are found persuasive. Further, none of Fu, Lin, Zin or any of the other cited prior art of record, whether considered alone or in a combination obvious to a person having ordinary skill in the art, teach or suggest the “updating, by the estimation-model encoder, the attention mask by removing the occluded portion of the object based on an unoccluded portion of the object” limitations of claims 9 and 17, in view of all of the limitations of claims 9 and 17 which serve to support and provide patentably distinguishing context to the claimed attention mask.
Conclusion
Applicant's amendment necessitated any of the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights