DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Response to Preliminary Amendment
The preliminary amendment received 03/07/2023 has been entered.
35 USC § 102 (f) Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “an input data generation unit” and “a ground truth data generation unit” in claim 17. The “unit” is a generic placeholder and is coupled with a functional language “generate”.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3 and 8-17 are rejected under 35 U.S.C. 102 (a)(2) as being anticipated by Marsden (US 20230214458 A1).
Regarding to claim 1 (Original), Marsden discloses an information processing method implemented by a computer ([0083]: a computer graphics simulator automatically generates simulated hand poses and gesture sequences; the hand poses and gesture sequences are generated across a variety of simulation parameters; [0102]: the ground truth hand pose 500A is labeled with twenty-eight 3D joint locations 500B; Fig. 54; [0256]: the pose vector 5412 is computed by the simulator 4100 as the ground truth label corresponding to the simulated hand images 4106), the information processing method comprising:
generating a simulation image that serves as input data by computer graphics (Fig. 54; [0070]: a computer graphics simulator generates simulated hand images; Fig. 54; [0255]: the computer graphics simulator 4100 automatically generates simulated hand images along with precise hand position parameters; Fig. 54; [0256]: the simulator 4100 generates the simulated hand images);
generating label information based on parameter information that is used for a
simulation (Fig. 54; [0070]: the generated corresponding label is assigned or mapped to the images in the form of the ground truth 84 (28 × 3) dimensional pose vector; [0102]: convolutional neural network 101 is trained using labeled dataset 500A and 500B; Fig. 54; [0256]: the pose vector 5412 is computed by the simulator 4100 as the ground truth label corresponding to the simulated hand images 4106;
PNG
media_image1.png
450
576
media_image1.png
Greyscale
; [0281-0282]); and
generating ground truth data that includes the simulation image and the label
information (Fig. 54; [0070]: a computer graphics simulator generates simulated hand images (left and right, (1, r)); the corresponding label is assigned or mapped to the images in the form of the ground truth 84 (28 × 3) dimensional pose vector of 3D joint locations of twenty-eight (28) hand joints; Fig. 5B; [0102]: the ground truth hand pose 500A is labeled with twenty-eight 3D joint locations 500B;
PNG
media_image2.png
192
584
media_image2.png
Greyscale
; the actual output and the ground truth desired output are in the form of capsule hand models, skeleton hand models, volumetric hand models and/or mesh hand models, muscle hand models, each in 2D and/or 3D space; Fig. 54; [0256]: the generated corresponding label is assigned or mapped 5400 to the images 4106 in the form of the ground truth 84 (28 × 3) dimensional pose vector 5412 of 3D joint locations of twenty-eight (28) hand joints; [0265]: simulator 4100 automatically labels or maps each of the simulated unique hand positions and gestures sequences to corresponding ground truth hand position parameters like pose vector 5412.).
Regarding to claim 2 (Original), Marsden discloses the information processing method according to claim 1, further comprising:
generating an analysis model based on machine learning in which the input data and
the ground truth data are used as training data (Marsden; Fig. 4; [0102]: train 400 a convolutional neural network; the trained CNN is an analysis model; convolutional neural network 101 is trained using labeled dataset 500A and 500B in a wide assortment of representative input image patterns that are mapped to their intended output 406 of ground truth hand pose estimates 500A; the actual output and the ground truth desired output are in the form of capsule hand models, skeleton hand models, volumetric hand models and/or mesh hand models, muscle hand models, each in 2D and/or 3D space; [0120]: convolution layers 104 of convolutional neural network 101 serve as feature extractors; [0254]: the simulated hand images along with the labelled hand position parameters from corresponding simulations are used for training convolutional neural network 101).
Regarding to claim 3 (Original), Marsden discloses the information processing method according to claim 2, further comprising:
extracting the label information from a captured image by using the analysis model (Marsden; [0080]: a final location for each joint is calculated based on the plurality of estimates for a particular joint; [0106]: generate an output, i.e. prediction of twenty-eight (28) joint locations of a hand in 3D; [0120]: convolution layers 104 of convolutional neural network 101 serve as feature extractors; [0122]: the convolution layers 104 extracts global and local hand features; [0157]: different features of the hand pose are identified and are extracted by the convolution kernels);
extracting parameter information based on the label information (Marsden; [0102]: The twenty-eight joints include four joints for the thumb, five joints for each of the index, middle, ring and pinkie fingers and four joints for the wrist or arm; [0106]: generate an output, i.e. prediction of twenty-eight (28) joint locations of a hand in 3D; [0253]: different hand position parameters are used for labeling the ground truth feature vector);
generating a simulation image based on the parameter information (Marsden; [0255]: simulator 4100 receives a specification of a range of simulation parameters and uses the specification to automatically generate different combinations of hand images with varying values within the range; Fig. 54; [0256]: the simulated hand images (left and right, (l, r)) 4106 are generated by simulator 4100);
correcting the parameter information until a difference between the simulation image
and the captured image meets a condition that is set in advance (Marsden; Fig. 4; [0102]: train 400 a convolutional neural network; the trained CNN is an analysis model; convolutional neural network 101 is trained using labeled dataset 500A and 500B in a wide assortment of representative input image patterns that are mapped to their intended output 406 of ground truth hand pose estimates 500A; convolutional neural network 101 is adjusted 410 using back propagation 408 based on a comparison of the output 404 and the target 406 until the network output 404 matches the target 406; [0106]: an error 412 between the output prediction 404 and the desired target 406 is measured; [0120]: convolution layers 104 of convolutional neural network 101 serve as feature extractors; [0254]: the simulated hand images along with the labelled hand position parameters from corresponding simulations are used for training convolutional neural network 101); and
estimating information based on the parameter information that has been corrected
until the condition is met (Marsden; Fig. 1; [0080]: pose estimation 114 and hand model fitting 116; [0102]: convolutional neural network 101 is adjusted 410 using back propagation 408 based on a comparison of the output 404 and the target 406 until the network output 404 matches the target 406; [0163]: generate rough hand poses estimates; [0164]: output different rough pose estimates for the same input hand image; [0166]: compute three (3) accurate estimates of hand position parameters; [0199]: the estimates are analyzed at a joint level and a final location for each joint is calculated based on the plurality of estimates for a particular joint).
Regarding to claim 8 (Original), Marsden discloses the information processing method according to claim 2, wherein
the simulation image is a simulation image of the individual that is generated by using an individual image that is based on a morphological feature of the individual (Marsden; Fig. 11; [0157]: the learned convolution kernels applied locally 1100 to an input image on the left produce a convolved image on the right;
PNG
media_image3.png
412
670
media_image3.png
Greyscale
; [0199]: perform hand pose estimation 114 on a so-called “joint-by-joint” basis; [0276]: generate ground-truth simulated stereoscopic hand images (l, r) for gesture sequences using a computer graphic simulator), and
the label information is generated based on parameter information on the individual
that is applied to the individual model (Marsden; [0080]: a final location for each joint is calculated based on the plurality of estimates for a particular joint; [0162]: expert networks 112 generate, as output, estimates of “hand position parameters; these hand position parameters are in the form of joint location models, joint angel models, capsule models, skeleton models, volumetric models and/or mesh models, muscle hand models, each in 2D and/or 3D space; [0281]: an 84 (28 × 3) dimensional pose vector of 3D joint locations of twenty-eight (28) hand joints is computed using the computer graphic simulator; ).
Regarding to claim 9 (Original), Marsden discloses the information processing method according to claim 8, further comprising:
extracting label information on each of individuals from a captured image by using
the analysis model (Marsden; [0120]: convolution layers 104 of convolutional neural network 101 serve as feature extractors; [0122]: the convolution layers 104 extracts global and local hand features; [0157]: different features of the hand pose are identified and are extracted by the convolution kernels; [0162]: expert networks 112 generate, as output, estimates of “hand position parameters; these hand position parameters are in the form of joint location models, joint angel models, capsule models, skeleton models, volumetric models and/or mesh models, muscle hand models, each in 2D and/or 3D space; Fig. 33; [0221]: for each individual hand joint, simultaneously identify outliers and inliers in the second set of estimates based on the similarity measure);
extracting parameter information on an individual based on the label information on
the individual, for each of the individuals (Marsden; [0217]: a first set of estimates of hand position parameters are received from multiple generalist and/or specialist neural networks for each of a plurality of hand joints; Fig. 33; [0221]: for each individual hand joint, simultaneously identify outliers and inliers in the second set of estimates based on the similarity measure; [0253]: different hand position parameters are used for labeling the ground truth feature vector); and
identifying each of the individuals based on the parameter information on each of the
individuals (Marsden; [0157]: represent different features of the hand pose identified and extracted by the convolution kernels; [0170]: identify similarities between portions of the poses or pose-types within the pose space 1700 in order to determine whether the poses or pose-types are characterized as forming a cluster; [0221]: for each individual hand joint, simultaneously identifying outliers and inliers in the second set of estimates based on the similarity measure).
Regarding to claim 10 (Original), Marsden discloses the information processing method according to claim 9, wherein the identifying includes
generating a simulation image of the individual based on the parameter
information on the individual (Marsden; [0230]: the individual master and expert pose estimates are shown inside each covariance in different colors; [0253]: different hand position parameters are used for labeling the ground truth feature vector),
correcting the parameter information on the individual until a difference
between the simulation image and the captured image of the individual meets a condition that
is set in advance (Marsden; Fig. 4; [0102]: train 400 a convolutional neural network; the trained CNN is an analysis model; convolutional neural network 101 is trained using labeled dataset 500A and 500B in a wide assortment of representative input image patterns that are mapped to their intended output 406 of ground truth hand pose estimates 500A; convolutional neural network 101 is adjusted 410 using back propagation 408 based on a comparison of the output 404 and the target 406 until the network output 404 matches the target 406; [0107]: an algorithm performs backward propagation of errors by means of gradient descent; [0115]: convolutional neural network 101 uses a gradient descent optimization to compute the error across all the layers; [0120]: convolution layers 104 of convolutional neural network 101 serve as feature extractors; [0168]: the hand position parameters predicted by both the master networks 110 and expert networks 112 are used to generate the final hand pose estimation; [0254]: the simulated hand images along with the labelled hand position parameters from corresponding simulations are used for training convolutional neural network 101), and
identifying the individual based on the parameter information on the
individual that has been corrected until the condition is met (Marsden; [0157]: represent different features of the hand pose identified and extracted by the convolution kernels; [0170]: identify similarities between portions of the poses or pose-types within the pose space 1700 in order to determine whether the poses or pose-types are characterized as forming a cluster; [0221]: for each individual hand joint, simultaneously identifying outliers and inliers in the second set of estimates based on the similarity measure).
Regarding to claim 11 (Original), Marsden discloses the information processing method according to claim 10, wherein the parameter information on the individual includes parameter information on a muscle parameter that is applied to a skeletal muscle model of the individual and parameter information indicating a movement feature value of the individual (Marsden; [0102]: the actual output and the ground truth desired output are in the form of capsule hand models, skeleton hand models, volumetric hand models and/or mesh hand models, muscle hand models, each in 2D and/or 3D space; [0162]: these hand position parameters are in the form of joint location models, joint angel models, capsule models, skeleton models, volumetric models and/or mesh models, muscle hand models, each in 2D and/or 3D space).
Regarding to claim 12 (Currently Amended), Marsden discloses the information processing method according to claim 9, further comprising:
detecting an abnormality of the individual by applying the captured image of the
individual to an abnormal behavior model (Marsden; [0076]: the predicted output is compared to the actual output; [0102]: comparison of the output 404 and the target 406 until the network output 404 matches the target 406; [0106]: an error 412 between the output prediction 404 and the desired target 406 is measured; [0115]: convolutional neural network 101 uses a gradient descent optimization to compute the error across all the layers; [0224]: minimize approximation error between corresponding 3D joint estimates).
Regarding to claim 13 (Currently Amended), Marsden discloses the information processing method according to claim 8, wherein
the generating the ground truth data includes
generating an image element for identifying the individual based on location
information on the individual that is used for the simulation (Marsden; Fig. 33; [0221]: for each individual hand joint, simultaneously identify outliers and inliers in the second set of estimates based on the similarity measure; [0255]: simulator 4100 receives a specification of a range of simulation parameters and uses the specification to automatically generate different combinations of hand images with varying values within the range; Fig. 54; [0256]: the simulated hand images (left and right, (l, r)) 4106 are generated by simulator 4100),
generating a corrected image by adding the image element to the simulation
image (Marsden; Fig. 4; [0102]: train 400 a convolutional neural network; the trained CNN is an analysis model; convolutional neural network 101 is trained using labeled dataset 500A and 500B in a wide assortment of representative input image patterns that are mapped to their intended output 406 of ground truth hand pose estimates 500A; convolutional neural network 101 is adjusted 410 using back propagation 408 based on a comparison of the output 404 and the target 406 until the network output 404 matches the target 406; [0120]: convolution layers 104 of convolutional neural network 101 serve as feature extractors; Fig. 36; [0237]: mageRects 3600 is fitted on an ImagePatch; [0254]: the simulated hand images along with the labelled hand position parameters from corresponding simulations are used for training convolutional neural network 101), and
generating the ground truth data by adding the label information to the
corrected image (Marsden; Fig. 54; [0070]: a computer graphics simulator generates simulated hand images (left and right, (1, r)); the corresponding label is assigned or mapped to the images in the form of the ground truth 84 (28 × 3) dimensional pose vector of 3D joint locations of twenty-eight (28) hand joints; Fig. 5B; [0102]: the ground truth hand pose 500A is labeled with twenty-eight 3D joint locations 500B;
PNG
media_image2.png
192
584
media_image2.png
Greyscale
; Fig. 36; [0237]: mageRects 3600 is fitted on an ImagePatch in yellow color;
PNG
media_image4.png
312
590
media_image4.png
Greyscale
; Fig. 54; [0256]: the generated corresponding label is assigned or mapped 5400 to the images 4106 in the form of the ground truth 84 (28 × 3) dimensional pose vector 5412 of 3D joint locations of twenty-eight (28) hand joints; [0265]: simulator 4100 automatically labels or maps each of the simulated unique hand positions and gestures sequences to corresponding ground truth hand position parameters like pose vector 5412).
Regarding to claim 14 (Original), Marsden discloses the information processing method according to claim 13, wherein the image element includes one of a frame element for enclosing the individual and a color element for identifying the individual by color (Marsden; [0156]: pre-processing include noise reduction, color space conversion, image scaling and Gaussian pyramid; Fig. 34; [0230]: the individual master and expert pose estimates are shown inside each covariance in different colors and the final fitted hand 5200 is shown in pale yellow-green; Fig. 36; [0237]: mageRects 3600 is fitted on an ImagePatch in yellow color;
PNG
media_image4.png
312
590
media_image4.png
Greyscale
; [0239]: the previous frame’s fitted hand model (multi-colored ellipses/ellipsoid) is extrapolated 3700).
Regarding to claim 15 (Currently Amended), Marsden discloses the information processing method according to claim 13, further comprising:
adding the image element to each of the individuals that appear in the captured image
by applying the captured image to the analysis model (Marsden; Fig. 11; [0157]: the learned convolution kernels applied locally 1100 to an input image on the left produce a convolved image on the right that is robust to the background and the clutter, i.e., ignores the background and the clutter and only extracts the hand features; [0290]: an optional video projector 5620 projects an image of a page from a virtual book object superimposed upon a real world object); and
individually tracing each of the individuals based on the image element that is added
to each of the individuals (Marsden; Fig. 11; [0157]: the learned convolution kernels applied locally 1100 to an input image on the left produce a convolved image on the right that is robust to the background and the clutter, i.e., ignores the background and the clutter and only extracts the hand features; [0290]: the back side of hand 5614 is projected to the user, so that the scene looks to the user as if the user is looking at their own hand).
Regarding to claim 16 (Currently Amended), Marsden discloses a non-transitory storage medium that stores a program that causes a computer to execute ([0083]: a computer graphics simulator automatically generates simulated hand poses and gesture sequences; the hand poses and gesture sequences are generated across a variety of simulation parameters; [0102]: the ground truth hand pose 500A is labeled with twenty-eight 3D joint locations 500B; Fig. 54; [0256]: the pose vector 5412 is computed by the simulator 4100 as the ground truth label corresponding to the simulated hand images 4106; Fig. 57; [0294]: store instructions to be executed by processor 5702; memory 5704 contains instructions; [0296]: processor 5702 is a general-purpose microprocessor; [0298]: Instructions defining mocap program 5714 are stored in memory 5704):
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 16.
Regarding to claim 17 (Original), Marsden discloses an information processing apparatus ([0083]: a computer graphics simulator automatically generates simulated hand poses and gesture sequences; the hand poses and gesture sequences are generated across a variety of simulation parameters; [0102]: the ground truth hand pose 500A is labeled with twenty-eight 3D joint locations 500B; Fig. 54; [0256]: the pose vector 5412 is computed by the simulator 4100 as the ground truth label corresponding to the simulated hand images 4106; Fig. 57; [0294]: store instructions to be executed by processor 5702; memory 5704 contains instructions; [0296]: processor 5702 is a general-purpose microprocessor; [0298]: Instructions defining mocap program 5714 are stored in memory 5704) comprising:
an input data generation unit (Fig. 57; [0294]: a presentation input interface); and
a ground truth data generation unit (Fig. 57; [0294]: processor ).
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 17.
Claims 4-7 are rejected under 35 U.S.C. 103 as being unpatentable over Marsden (US 20230214458 A1) and in view of Nougaret (US 20020093503 A1 ).
Regarding to claim 4 (Original), Marsden discloses the information processing method according to claim 2, wherein
the simulation image is a simulation image of a school that is generated by using a
school model that is based on a statistical property of the school (Marsden; [0083]: automatically generates simulated hand poses and gesture sequences in the order of 100,000 and a billion; Fig. 5A; [0102]: the target hand pose estimates 500A are labeled with twenty-eight (28) joint locations of the hand in three-dimensions (3D); Fig. 40A; [0250]: skeleton hand models are fitted to estimated joint covariances interacting with and manipulating 4000A, 4000B and 4000C virtual objects;
PNG
media_image5.png
406
694
media_image5.png
Greyscale
; joins of two hands are a group of joins; [0253]: generate between hundred thousand (100,000) and one billion (1,000,000,000) simulated hand positions and gesture sequences with varying hand-anatomy and hand-background simulations; Fig. 41; [0255]: automatically generate simulated hand images along with precise hand position parameters; simulated hand images 4106), and
the label information is generated based on parameter information on the school that
is applied to the school model (Marsden; [0256]: the corresponding label assigned or mapped 5400 to the images 4106 in the form of the ground truth 84 (28 × 3) dimensional pose vector 5412 of 3D joint locations of twenty-eight (28) hand joints; [0257]: simulator 4100 defines pose vector 5412 in terms of angles of skeleton model; [0258]: the defined simulation parameters are used to compute the ground truth hand position parameters of the pose vector 5412).
Marsden fails to explicitly disclose a school model that is based on a statistical property of the school.
In same field of endeavor, Nougaret teaches a school model that is based on a statistical property of the school ([0031]: six fishes are contained in a group; [0041]: a group of fishes or animals is animated by means of a dynamic simulation and on the basis of a physical model; the physical characteristics associated with the motion of the members of the group are reflected in the animation; [0052]: a coordination algorithm for controlling individual members of the group frame by frame; Fig. 3; [0081]: a large number of airplanes fly as a group; [0117]: a clustering technique is used in a simulation of a many-body (n-body) problem; [0132]: perform a computer simulation to check whether the algorithm is trapped; [0138]: a modeling technique, a simulation technique, a content authoring tool, a run-time engine, etc., have been developed; [0149]: linearly simulates the dynamic behavior around a certain operating point in a state space; [0201]: the members are moved along the shortest distances to the locations in the target layout).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Marsden to include a school model that is based on a statistical property of the school as taught by Nougaret. The motivation for doing so would have been to animate a group of fishes or animals by means of a dynamic simulation and on the basis of a physical model; to improve the role assignment technique as taught by Nougaret in paragraphs [0041] and [0164].
Regarding to claim 5 (Original), Marsden and Nougaret discloses the information processing method according to claim 4, further comprising:
extracting the label information from a captured image of the school by using the
analysis model (Marsden; Fig. 5B; [0102]: the target hand pose estimates 500A are labeled with twenty-eight (28) joint locations of the hand in three-dimensions (3D); [0253]: each simulation is labeled with fifteen (15) to forty-five (45) hand position parameters such as 3D joint locations; Fig. 54; [0256]: the corresponding label assigned or mapped 5400 to the images 4106 in the form of the ground truth 84 (28 × 3) dimensional pose vector 5412 of 3D joint locations of twenty-eight (28) hand joints; Fig. 55; [0277]: stereoscopic hand boundaries are extracted and aligned with hand centers);
extracting parameter information on the school based on the label information (Marsden; Fig. 5A; [0102]: the twenty-eight joints include four joints for the thumb, five joints for each of the index, middle, ring and pinkie fingers and four joints for the wrist or arm;
PNG
media_image6.png
476
548
media_image6.png
Greyscale
; [0106]: generate an output, i.e. prediction of twenty-eight (28) joint locations of a hand in 3D; [0253]: different hand position parameters are used for labeling the ground truth feature vector; [0278]: generate translated, rotated and scaled variants of the stereoscopic hand boundaries); and
estimating number of individuals that are included in the school based on the
parameter information on the school (Marsden; Fig. 5B; [0102]: the target hand pose estimates 500A are labeled with twenty-eight (28) joint locations of the hand in three-dimensions (3D); one implementation of the ground truth hand pose 500A with twenty-eight joint locations in 3D is graphically illustrated in Fig. 5A and Fig. 5B;
PNG
media_image7.png
474
580
media_image7.png
Greyscale
; [0281]: an 84 (28 × 3) dimensional pose vector of 3D joint locations of twenty-eight (28) hand joints is computed using the computer graphic simulator).
Regarding to claim 6 (Original), Marsden and Nougaret discloses the information processing method according to claim 5, wherein the estimating includes
generating a simulation image of the school based on the parameter
information on the school (Marsden; [0255]: simulator 4100 receives a specification of a range of simulation parameters and uses the specification to automatically generate different combinations of hand images with varying values within the range; Fig. 54; [0256]: the simulated hand images (left and right, (l, r)) 4106 are generated by simulator 4100),
correcting the parameter information on the school until a difference between
a simulation image and the captured image of the school meets a condition that is set in
advance (Marsden; Fig. 4; [0102]: train 400 a convolutional neural network; the trained CNN is an analysis model; convolutional neural network 101 is trained using labeled dataset 500A and 500B in a wide assortment of representative input image patterns that are mapped to their intended output 406 of ground truth hand pose estimates 500A; convolutional neural network 101 is adjusted 410 using back propagation 408 based on a comparison of the output 404 and the target 406 until the network output 404 matches the target 406; [0106]: an error 412 between the output prediction 404 and the desired target 406 is measured; [0120]: convolution layers 104 of convolutional neural network 101 serve as feature extractors; [0254]: the simulated hand images along with the labelled hand position parameters from corresponding simulations are used for training convolutional neural network 101), and
estimating number of the individuals based on the parameter information on
the school that has been corrected until the condition is met (Marsden; Fig. 5B; [0102]: the target hand pose estimates 500A are labeled with twenty-eight (28) joint locations of the hand in three-dimensions (3D); one implementation of the ground truth hand pose 500A with twenty-eight joint locations in 3D is graphically illustrated in Fig. 5A and Fig. 5B;
PNG
media_image7.png
474
580
media_image7.png
Greyscale
; [0163]: generate rough hand poses estimates; [0164]: output different rough pose estimates for the same input hand image; [0166]: compute three (3) accurate estimates of hand position parameters).
Regarding to claim 7 (Currently Amended), Marsden and Nougaret discloses the information processing method according to claim 5, wherein the simulation image of the school is a simulation image of the school at a time of feeding (Marsden; [0238]: tracking is performed by updating each ImageRect across frames using prior hand movements to extrapolate the ImageRect forward in time; [0239]: the previous frame’s fitted hand model is extrapolated 3700 into the current frame’s timestamp; [0249]: track in real time the most subtle and minute hand gestures, along with most extreme hand gestures; [0259]: the real gesture recognition system is a 3D time-of-flight camera; [0268]: examine and investigate hand poses or frames of a gesture sequence at a given timestamp; edit the different simulation parameters discussed infra at a given hand pose, frame or timestamp to generate and store a new simulated gesture sequence, variant simulated gesture sequence, morphed simulated gesture sequence or altered or modified simulated gesture sequence).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights