DETAILED ACTION
Response to Amendment
Applicant’s amendments filed on 13 April 2026 have been entered. Claims 1-5, 10, 12-14, and 19 were amended. Hence, claims 1-20 are pending in the application, with claims 1 and 12 being independent.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 9, 10, 12, 14, 18 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cho et al. (US 20210264144 A1), referred herein as Cho in view of Kim et al. (US 20240013564 A1), referred herein as Kim.
Regarding Claim 1, Cho in view of Kim teaches a method comprising (Cho Abst: System and method for extracting human pose information from an image, comprising a feature extractor connected to a database; [0076] The memory 804 optionally includes one or more storage devices remotely located from the CPU(s) 802. The memory 804, or alternately the non-volatile memory device(s) within the memory 804, comprises a non-transitory computer readable storage medium.):
for each image in a set of images (Cho [0007] a feature extractor for extracting human-related image features from the image):
storing a parametric representation of an object depicted in said each image, wherein the parametric representation comprises a set of parameter values for a set of parameters that describe the object (Cho [0050] The number of convolutional layers present in the intermediate feature extractors 120 and their parameters for each intermediate stage are tailored by the size of input image and target objects, i.e. humans, as described above. Each intermediate stage is trained using the dataset of reference images; [0071] the feature extractor 30 determines all the human-related image features that can be obtained from an image and stores them at each neural network layer in a tensor form);
training a neural network based on said each image and the parametric representation of the object depicted in said each image (Cho [0031] the data set of reference images comprises at least tens of thousands of images for training and may be qualified as being large scale; [0033] Each convolutional layer applies convolutional operation to its input data using trained kernel weights. The training of the weights of convolutional layers is performed by backpropagation technique using the dataset of reference images);
Cho disclosed trained neural network (Cho [0033] Each convolutional layer applies convolutional operation to its input data using trained kernel weights; [0050] Each intermediate stage is trained using the dataset of reference images), but does not teach inputting an image into the neural network.
In viewing of Kim, the prior art teaches
after training the neural network, inputting an image into the neural network (Kim [0044] After a number (e.g., thousands) of iterations of the training procedure, CNN 304, transformer encoder 308, and transformer decoder 314 may be trained to receive images and generate output embeddings that relate to bounding boxes of objects in the images; [0076] detection of objects in content signal 602 (e.g., an image); FIG.6A.610 & 614: pretrained encoder & decoder);
Cho in view of Kim further teaches
based on inputting the image into the neural network, generating an output that comprises a particular parametric representation of a particular object depicted in the image (Kim [0078] According to an embodiment, feature pyramid network 628 may comprise a feature pyramid network (FPN) that may receive a single-scale image of an arbitrary size as an input (e.g., in an input tensor for a neural network), and output proportionally sized feature maps at multiple levels),
wherein the particular parametric representation comprises a set of output parameter values for the set of parameters (Kim [0079] feature pyramid network 628 may comprise one or more neural networks to receive parameters and/or values from extractor 604 and/or decoder 614 formatted into one or more input tensors to be processed by feature pyramid network 628. Based, at least in part, on the one or more input tensors of the feature pyramid network 628, feature pyramid network 628 may compute one or more output tensors);
wherein the method is performed by one or more computing devices (Cho [0076] The processing module 800 typically includes one or more Computer Processing Units (CPUs) and/or Graphic Processing Units (GPUs) 802; Kim [0002] machine-learning devices).
Kim discloses an apparatus and method for using one or more computing devices to implement one or more encoding and/or decoding techniques. Kim is analogous to the present patent application.
It would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cho to incorporate the teachings of Kim, and apply processes for training encoder and/or decoder parameters for object detection and/or classification into the system and method for extracting human pose information from an image.
Doing so would yield visual representations having an associated accuracy that approaches a level of accuracy enabled using fully supervised learning operations on large computer vision downstream tasks.
Regarding Claim 4, Cho in view of Kim teaches method of Claim 1, and further teaches wherein the set of parameters is for a first dimension in a multi-dimensional space, the method further comprising:
for each image in the set of images, storing a second set of parameter values for a second set of parameters that describes the object depicted in said each image (Cho [0028] The system 10 is configured to first extract from the images human-related image features that are learned by an image dataset and determine the human pose information from the extracted human-related image features; [0050] a stack of 2D joint heat map in which the human joints in an image are marked in the same location may be generated using 2D joint locations; claim 1. the 3D body skeleton detector and the facial keypoints detector is provided with a second convolutional neural network (CNN) architecture including a second plurality of CNN layers);
wherein the second set of parameters is for a second dimension, in the multi- dimensional space, that is different than the first dimension (Cho [0068] The 3D body skeleton detector 90 is configured for estimating 3D coordinates of human body joints from a single image. The 3D body skeleton detector 90 receives human-related image feature tensors and estimates normalized 3D coordinates of a human body detected in an image. The post-processing module 620 is configured for mapping the normalized 3D locations into image and real-world spaces). The second CNN performs the identical functions for the 2D object in a different view for a 3D object.
Regarding Claim 9, Cho in view of Kim teaches method of Claim 1, and further teaches wherein the neural network comprises an embedding layer, a set of convolution layers, and a set of fully connected layers (Cho [0032] The system 10 uses convolutional neural network (CNN) architecture for robust estimation of the pose information. CNNs are composed of convolutional neural network layers; [0050] he model is trained by adjusting weight and bias values by repeating forward and backward propagation process throughout the connected layers in the neural networks; Kim [0061] Such a whole collection of features combined with positional embedding may be processed by a decoder (e.g., decoder 414 or 415)).
Regarding Claim 10, Cho in view of Kim teaches method of Claim 1, and further teaches further comprising, prior to training the neural network:
for each image in the set of images:
generating a set of points that describe the object in said each image (Cho [0007] a facial keypoints detector for determining facial keypoints from the human-related image features, wherein each one of the 2D body skeleton detector, the body silhouette detector, the hand silhouette detector, the hand skeleton detector, the 3D body skeleton detector and the facial keypoints detector is provided with a second convolutional neural network (CNN) architecture including a second plurality of CNN layers),
generating the set of parameter values by applying a parametric fitting to the set of points (Cho [0008] the feature extractor comprises: a low-level feature extractor for extracting low-level features from the image; and an intermediate feature extractor for extracting intermediate features, the low-level features and the intermediate features forming together the human-related image features).
Regarding Claims 12, 14, 18 and 19, Cho in view of Kim teaches one or more non-transitory storage media storing instructions. The metes and bounds of the limitations of the claims substantially correspond to the elements set forth in claims 1 and 4, 9 and 10; thus they are rejected on similar grounds and rationale as their corresponding limitations.
Claim(s) 2, 3, 5-7, 13, 15 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cho et al. (US 20210264144 A1), referred herein as Cho in view of Kim et al. (US 20240013564 A1), referred herein as Kim and Chen et al. (US 20200377087 A1), referred herein as Chen.
Regarding Claim 2, Cho in view of Kim teaches the method of Claim 1, but does not teach the claimed limitations herein. However, Chen teaches wherein the object is a track boundary of a track that one or more moving objects traverse (Chen [0023] Images from the cameras can be processed to detect objects such as streetlights, stop signs, lines or borders of one or more lanes of a road).
Chen discloses an autonomous vehicle (AV) is automatically navigated within a a driving lane. Chen is analogous to the present patent application.
It would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cho to incorporate the teachings of Chen, and apply the lane line data detection method into the system and method for extracting human pose information from an image.
Doing so would allow the AV to be navigated along the road based on the trajectory points.
Regarding Claim 3, Cho in view of Kim and Chen teaches method of Claim 2, and further teaches wherein the track boundary is a first track boundary, the method further comprising:
storing, for each image in the set of images, a second set of parameter values for a second set of parameters that describes a second track boundary in said each image (Cho [0007] a 3D body skeleton detector for determining 3D body skeleton from the human-related image features; and a facial keypoints detector for determining facial keypoints from the human-related image features, wherein each one of the 2D body skeleton detector, the body silhouette detector, the hand silhouette detector, the hand skeleton detector, the 3D body skeleton detector and the facial keypoints detector is provided with a second convolutional neural network (CNN) architecture including a second plurality of CNN layers; Chen Abst: A second parametric curve can be matched to a location of a second lane line detected from the lane line data);
wherein training the neural network is also based on the second set of parameter values of each image (Cho [0013] at least one of the first and second architecture comprises a deep CNN architecture). The second CNN performs the identical functions for the 2D object in a different view for a 3D object.
Regarding Claim 5, Cho in view of Kim teaches method of Claim 4. However Chen teaches wherein the set of parameters is for a first polynomial function and the second set of parameters is a second polynomial function (Chen [0052] Parametric curve functions to be matched to each lane line can be created at step 530… the functions y=f.sub.i(x, p.sub.1.sup.i, p.sub.2.sup.i, . . . , p.sub.n.sup.i) can be third order polynomials of x). The same motivation as claim 2 applies here.
Regarding Claim 6, Cho in view of Kim teaches method of Claim 1. However Chen teaches further comprising: based on the set of parameter values, determining a lateral position of a moving object that is associated with the image (Chen [0031] Planning module 212 may receive and process data and information received from the perception module 220 to plan actions for the AV. The action planning may include adjusting a lateral position of an AV navigating a lane, accelerating within a current lane, braking within a current lane, and other actions). The same motivation as claim 2 applies here.
Regarding Claim 7, Cho in view of Kim and Chen teaches method of Claim 6, and further teaches wherein determining the lateral position of the moving object comprises:
determining a position of the particular object based on the set of output parameter values (Chen [0058] First, a plurality of actions to navigate from the current position and heading through next set of sampled coordinates are generated at step 710. The sampled coordinates represent a vector of trajectory points along a center lane line curve as described by a function);
generating a difference between a current position of the moving object and the position of the particular object (Chen [0058] The generated action may be indicative on an angle at which a wheel of the AV is to be steered at the current position and one or more subsequent sampled locations).
Regarding Claims 13, 15 and 16, Cho in view of Kim teaches the one or more non-transitory storage media of Claim 12. The metes and bounds of the limitations of the claims substantially correspond to the elements set forth in claims 3, 6 and 7; thus they are rejected on similar grounds and rationale as their corresponding limitations.
Claim(s) 8, 11, 17 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cho et al. (US 20210264144 A1), referred herein as Cho in view of Kim et al. (US 20240013564 A1), referred herein as Kim and VINEET et al. (US 20220383648 A1), referred herein as VINEET.
Regarding Claim 8, Cho in view of Kim teaches the method of Claim 1, but does not teach the claimed limitations herein. However, VINEET teaches wherein training the neural network comprises minimizing a cost function that is based on a distance between a predicted position of the object in said each image and an actual position of the object in said image (VINEET [0021] As another example, the 3D images in a given training input may be images that were captured simultaneously by different image capture units (such as a stereoscopic pair of image capture units) having a certain geometric relationship to each other (e.g. located a fixed distance apart). In this case, the expected differences in the position and/or orientation of the predicted 3D objects are determined by the geometric relationship between the image capture units, and the cost function may in that case penalize deviations from those expected differences).
VINEET discloses a method of training a 3D structure detector to detect 3D structure in 3D structure representation. VINEET is analogous to the present patent application.
It would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified Cho to incorporate the teachings of VINEET, and apply the cost function for differences into the system and method for extracting human pose information from an image.
Doing so would able to perform 2D bounding box detection with exceptional accuracy in almost any practical context.
Regarding Claim 11, Cho in view of Kim teaches method of Claim 10. However VINEET teaches further comprising:
defining a size of a bounding box (VINEET [0017] A predicted 3D object may be represented as one or more of: a 3D location/position, a 3D orientation/pose, and a size (extend in 3D space). A 3D bounding box or other 3D boundary object may encode all three (position, orientation and size));
for each image in the set of images:
projecting the bounding box onto a moving object that is associated with said each image (VINEET [0048] projecting the predicted 3D boundary object into a 2D image plane;);
wherein generating the set of points is based on a coordinate space that is defined by the bounding box (VINEET [0117] These define the x-y coordinates of the upper left (ul) and lower right (lr) corner points of the predicted 2D bounding box 504). The same motivation as claim 8 applies here.
Regarding Claims 17 and 20, Cho in view of Kim teaches the one or more non-transitory storage media of Claim 12. The metes and bounds of the limitations of the claims substantially correspond to the elements set forth in claims 8 and 11; thus they are rejected on similar grounds and rationale as their corresponding limitations.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1 and 12 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Samantha (Yuehan) Wang whose telephone number is (571)270-5011. The examiner can normally be reached Monday-Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached on (571)272-7440. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Samantha (YUEHAN) WANG/
Primary Examiner
Art Unit 2617
Prosecution Insights