DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) were submitted on 10/29/2024 and 4/2/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Objections
Claims 1 and 7-9 are objected to because of the following informalities:
Claim 1, line 7, Examiner suggests inserting a colon after “device comprises”.
Claims 1 and 7-9, commas are used to separate the claim elements. Examiner suggests amending to use semicolons instead.
Claim 8, line 8, Examiner suggests inserting a colon after “method includes”.
Claim 9, lines 6-7, Examiner suggests inserting a colon after “to execute”.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Regarding claim 8, “a coordinate transformation step”, “a determination step”, and “an estimation step” are all interpreted as programs or processes stored on a processor, as described in Fig. 4 and ¶0022, “programs corresponding to these units are loaded in a CPU”.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
“coordinate transformation unit”, “determination unit”, and “estimation unit” in claims 1-7.
“skeleton estimation unit” in claims 4-6.
“article-take-out unit” in claim 5.
“association unit” in claims 5 and 6.
“coordinate transformation procedure”, “determination procedure”, and “estimation procedure” in claim 9.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Regarding claims 1-7, “coordinate transformation unit”, “determination unit”, and “estimation unit” are all interpreted as programs or processes stored on a processor, as described in Fig. 4 and ¶0022, “programs corresponding to these units are loaded in a CPU”.
Regarding claims 4-6, “skeleton estimation unit” is interpreted as a program or process stored on a processor, as described in Fig. 14 and ¶0058, “programs corresponding to these units are loaded in a CPU”.
Regarding claims 5, “article-take-out unit” is interpreted as a program or process stored on a processor, as described in Fig. 14 and ¶0058, “programs corresponding to these units are loaded in a CPU”.
Regarding claims 5 and 6, “association unit” is interpreted as a program or process stored on a processor, as described in Fig. 14 and ¶0058, “programs corresponding to these units are loaded in a CPU”.
Regarding claims 9, “coordinate transformation procedure”, “determination procedure”, and “estimation procedure” are all interpreted as programs or processes stored on a processor, as described in Fig. 4 and ¶0022, “programs corresponding to these units are loaded in a CPU”.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 3 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 3, the use of “equal to” on line 6 regarding points p’1 and p’2 is indefinite as it is unclear how the equivalency is determined. The points are defined relative to two different origins, C1 and C2, respectively, and do not appear to be intended to have the same absolute coordinates as illustrated in Fig. 7. For purposes of examination, Examiner will interpret “equal to” as to mean the same relative position from C1 and C2.
Further, both β1 and β2 are defined by “the origin of the stereo spherical image coordinates”. However, there are two origins recited prior in the claim, C1 and C2. For the purposes of examination, Examiner will consider β1 to be formed in part by a straight line from p’1 to C1 and β2 by a straight line from p’2 to C2.
Claim Rejections - 35 USC § 101
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed
to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four
categories of patent eligible subject matter because “a computer program product” as claimed in claim 9 is software per se.
Positive Statement Regarding - 35 USC § 101
Regarding claims 1-8, the Examiner’s 35 U.S.C. 101 analysis recognizes that the claimed subject matter is directed to a practical application of a technical solution. The claimed elements, taken as a whole, improve the functioning of 3D position estimation systems by improving the efficiency using stereo spherical coordinates when determining an epipolar constraint, see ¶0005-0006 and ¶0093. Because the claims recite specific, claimed steps and structural elements that produce a tangible technical result, they are not directed to an abstract idea absent additional inventive concept limitations. Accordingly, the record supports a positive 101 determination for the present claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4, and 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 2021/0183098) in view of Liu et al. (Liu, Shuai, et al. "3d spherical panoramic epipolar line based on essential matrix." IEEE Access 8 (2020): 192165-192176) (hereafter, “Liu”).
Regarding claim 1, Huang discloses a position estimation system comprising: three imaging devices (Fig. 2A; ¶0043, system 1 comprises a plurality of imaging devices; ¶0048, three views V1-V3); and a position estimation device configured to estimate a position of an object in a three-dimensional space (¶0020, compute a plurality of estimated three-dimensional, 3D, positions in a scene), based on images captured by the imaging devices (¶0044, The images captured by the imaging devices), wherein the three imaging devices are arranged such that imaging positions of the imaging devices form a triangle (Fig. 2A; Fig. 2A illustrates three imaging devices, V1-V3, arranged in a triangle), and the position estimation device comprises a coordinate transformation unit configured to respectively transform coordinate points of the object in image coordinates of the imaging devices to coordinate points in [stereo spherical] image coordinates (¶0048, a respective 3D position 10A-10C in the scene coordinate system 30 may be computed based on the detected positions of the objects O1-O3 in the local coordinate system 32 of the respective view V1-V3. Examiner considers the local coordinates of the views to be the image coordinates), a determination unit configured to determine whether or not the coordinate points of two of the imaging devices in the stereo [spherical image] coordinates satisfy an epipolar constraint (¶0054, based on epipolar geometry. Examiner considers epipolar geometry to imply satisfying the epipolar constraint), and an estimation unit configured to estimate that the coordinate points, which are determined by the determination unit to satisfy the epipolar constraint, correspond to a coordinate position of the object (¶0054, the 3D position is calculated by use of conventional triangulation, for example based on epipolar geometry).
However, Huang fails to explicitly disclose stereo spherical image coordinates.
Liu teaches stereo spherical image coordinates (Eqn. 1, 2; Fig. 1; Page 192166, Left column, last paragraph, the spherical angle parameters (φ; θ) of the pixel coordinates).
Both Huang and Liu are analogous to the claimed invention because both are directed towards determining a 3D position based of an object from multiple camera views. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the spherical coordinates of Liu into the position determination method of Huang. The suggestion/motivation for doing so would have been to increase versatility, as suggested by Liu at Page 192166, left column, paragraph 2, the proposed method is more versatile than the photogrammetric method.
This method of improving Huang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Liu.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Huang with the teachings of Liu to obtain the invention as specified in claim 1.
Regarding claim 2, in which claim 1 is incorporated, Huang discloses wherein the coordinate transformation unit transforms a coordinate point of the object in image coordinates captured by a first imaging device to a coordinate point p1 [in the stereo spherical image coordinates] (Fig. 2A; ¶0048, a respective 3D position 10A-10C in the scene coordinate system 30 may be computed based on the detected positions of the objects O1-O3 in the local coordinate system 32 of the respective view V1-V3. Examiner considers V1 and the corresponding point of any of objects O1-O3 as the ”p1”), transforms a coordinate point of the object in image coordinates captured by a second imaging device to a coordinate point p2 [in the stereo spherical image coordinates] (Fig. 2A; ¶0048, a respective 3D position 10A-10C in the scene coordinate system 30 may be computed based on the detected positions of the objects O1-O3 in the local coordinate system 32 of the respective view V1-V3. Examiner considers V2 and the corresponding point of any of objects O1-O3 as the ”p2”), and transforms a coordinate point of the object in image coordinates captured by a third imaging device to a coordinate point p3 [in the stereo spherical image coordinates] (Fig. 2A; ¶0048, a respective 3D position 10A-10C in the scene coordinate system 30 may be computed based on the detected positions of the objects O1-O3 in the local coordinate system 32 of the respective view V1-V3. Examiner considers V3 and the corresponding point of any of objects O1-O3 as the ”p3”).
However, Huang fails to explicitly disclose in the stereo spherical image coordinates.
Liu teaches in the stereo spherical image coordinates (Eqn. 1, 2; Fig. 1; Page 192166, Left column, last paragraph, the spherical angle parameters (φ; θ) of the pixel coordinates).
Both Huang and Liu are analogous to the claimed invention because both are directed towards determining a 3D position based of an object from multiple camera views. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the spherical coordinates of Liu into the position determination method of Huang. The suggestion/motivation for doing so would have been to increase versatility, as suggested by Liu at Page 192166, left column, paragraph 2, the proposed method is more versatile than the photogrammetric method.
This method of improving Huang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Liu.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Huang with the teachings of Liu to obtain the invention as specified in claim 2.
Regarding claim 4, in which claim 1 is incorporated, Huang discloses wherein the position estimation device further comprises a skeleton estimation unit configured to estimate a skeleton of a person, as the object (Fig. 1C, 1D; ¶0045, identifies one or more keypoints of one or more objects detected in the respective image … keypoints K1-K14 that may be detected for a human individual. Examiner considers K1-K14 as illustrated in Figs. 1C and 1D to be the skeleton of a person), imaged by each imaging device (¶0044, The images captured by the imaging devices), and the estimation unit estimates a coordinate position of a neck (Fig. 1C, 1D; ¶0045, the main keypoint K1 is indicated by an open circle and corresponds to the neck of the respective individual. Since the limitation is recited in the alternative, Examiner considers this citation to fully disclose the limitation) or a wrist in the skeleton estimated by the skeleton estimation unit.
Regarding claim 7, Huang discloses a position estimation device which estimates a position of an object in a three-dimensional space (¶0020, compute a plurality of estimated three-dimensional, 3D, positions in a scene), based on images captured by three imaging devices (Fig. 2A; ¶0043, system 1 comprises a plurality of imaging devices; ¶0048, three views V1-V3) arranged such that imaging positions of the imaging devices form a triangle (Fig. 2A; Fig. 2A illustrates three imaging devices, V1-V3, arranged in a triangle), the position estimation device comprising: a coordinate transformation unit configured to respectively transform coordinate points of the object in image coordinates of the imaging devices to coordinate points in [stereo spherical] image coordinates (¶0048, a respective 3D position 10A-10C in the scene coordinate system 30 may be computed based on the detected positions of the objects O1-O3 in the local coordinate system 32 of the respective view V1-V3. Examiner considers the local coordinates of the views to be the image coordinates), a determination unit configured to determine whether or not the coordinate points of two of the imaging devices in the [stereo spherical] image coordinates satisfy an epipolar constraint (¶0054, based on epipolar geometry. Examiner considers epipolar geometry to imply satisfying the epipolar constraint), and an estimation unit configured to estimate that the coordinate points, determined by the determination unit to satisfy the epipolar constraint, correspond to a coordinate position of the object (¶0054, the 3D position is calculated by use of conventional triangulation, for example based on epipolar geometry).
However, Huang fails to explicitly disclose stereo spherical image coordinates.
Liu teaches stereo spherical image coordinates (Eqn. 1, 2; Fig. 1; Page 192166, Left column, last paragraph, the spherical angle parameters (φ; θ) of the pixel coordinates).
Both Huang and Liu are analogous to the claimed invention because both are directed towards determining a 3D position based of an object from multiple camera views. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the spherical coordinates of Liu into the position determination method of Huang. The suggestion/motivation for doing so would have been to increase versatility, as suggested by Liu at Page 192166, left column, paragraph 2, the proposed method is more versatile than the photogrammetric method.
This method of improving Huang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Liu.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Huang with the teachings of Liu to obtain the invention as specified in claim 7.
Regarding claim 8, Huang discloses a position estimation method which is performed in a position estimation system including three imaging devices (Fig. 2A; ¶0043, system 1 comprises a plurality of imaging devices; ¶0048, three views V1-V3) and a position estimation device configured to estimate a position of an object in a three-dimensional space (¶0020, compute a plurality of estimated three-dimensional, 3D, positions in a scene) based on images captured by the imaging devices (¶0044, The images captured by the imaging devices), wherein the three imaging devices are arranged such that imaging positions of the imaging devices form a triangle (Fig. 2A; Fig. 2A illustrates three imaging devices, V1-V3, arranged in a triangle), and the position estimation method includes a coordinate transformation step in which the position estimation device respectively transforms coordinate points of the object in image coordinates of the imaging devices to coordinate points in [stereo spherical] image coordinates (¶0048, a respective 3D position 10A-10C in the scene coordinate system 30 may be computed based on the detected positions of the objects O1-O3 in the local coordinate system 32 of the respective view V1-V3. Examiner considers the local coordinates of the views to be the image coordinates), a determination step in which the position estimation device determines whether or not the coordinate points of two of the imaging devices in the [stereo spherical] image coordinates satisfy an epipolar constraint (¶0054, based on epipolar geometry. Examiner considers epipolar geometry to imply satisfying the epipolar constraint), and an estimation step in which the position estimation device estimates that the coordinate points, determined by the determination to satisfy the epipolar constraint, correspond to a coordinate position of the object (¶0054, the 3D position is calculated by use of conventional triangulation, for example based on epipolar geometry).
However, Huang fails to explicitly disclose stereo spherical image coordinates.
Liu teaches stereo spherical image coordinates (Eqn. 1, 2; Fig. 1; Page 192166, Left column, last paragraph, the spherical angle parameters (φ; θ) of the pixel coordinates).
Both Huang and Liu are analogous to the claimed invention because both are directed towards determining a 3D position based of an object from multiple camera views. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the spherical coordinates of Liu into the position determination method of Huang. The suggestion/motivation for doing so would have been to increase versatility, as suggested by Liu at Page 192166, left column, paragraph 2, the proposed method is more versatile than the photogrammetric method.
This method of improving Huang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Liu.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Huang with the teachings of Liu to obtain the invention as specified in claim 8.
Regarding claim 9, Huang discloses a computer program product (¶0082, A control program 51A comprising computer instructions is stored in the memory 52 and executed by the processing system 51 to perform any of the methods, operations, functions or steps exemplified in the foregoing) for position estimation which is used in a device configured to estimate a position of an object in a three-dimensional space (¶0020, compute a plurality of estimated three-dimensional, 3D, positions in a scene) based on images captured by three imaging devices (Fig. 2A; ¶0043, system 1 comprises a plurality of imaging devices; ¶0044, The images captured by the imaging devices; ¶0048, three views V1-V3) arranged such that imaging positions of the imaging devices form a triangle (Fig. 2A; Fig. 2A illustrates three imaging devices, V1-V3, arranged in a triangle), wherein the computer program product causes the device to execute a coordinate transformation procedure for respectively transforming coordinate points of the object in image coordinates of the imaging devices to coordinate points in [stereo spherical] image coordinates (¶0048, a respective 3D position 10A-10C in the scene coordinate system 30 may be computed based on the detected positions of the objects O1-O3 in the local coordinate system 32 of the respective view V1-V3. Examiner considers the local coordinates of the views to be the image coordinates), a determination procedure for determining whether or not the coordinate points of two of the imaging devices in the [stereo spherical] image coordinates satisfy an epipolar constraint (¶0054, based on epipolar geometry. Examiner considers epipolar geometry to imply satisfying the epipolar constraint), and an estimation procedure for estimating that the coordinate points, determined by the determination procedure to satisfy the epipolar constraint, correspond to a coordinate position of the object (¶0054, the 3D position is calculated by use of conventional triangulation, for example based on epipolar geometry).
However, Huang fails to explicitly disclose stereo spherical image coordinates.
Liu teaches stereo spherical image coordinates (Eqn. 1, 2; Fig. 1; Page 192166, Left column, last paragraph, the spherical angle parameters (φ; θ) of the pixel coordinates).
Both Huang and Liu are analogous to the claimed invention because both are directed towards determining a 3D position based of an object from multiple camera views. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the spherical coordinates of Liu into the position determination method of Huang. The suggestion/motivation for doing so would have been to increase versatility, as suggested by Liu at Page 192166, left column, paragraph 2, the proposed method is more
versatile than the photogrammetric method.
This method of improving Huang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Liu.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Huang with the teachings of Liu to obtain the invention as specified in claim 9.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 2021/0183098) in view of Liu et al. (Liu, Shuai, et al. "3d spherical panoramic epipolar line based on essential matrix." IEEE Access 8 (2020): 192165-192176) (hereafter, “Liu”) as applied to claim 1 above, and further in view of Singh et al. (US 2022/0262069) (hereafter, “Singh”) (IDS).
Regarding claim 5, Huang in view of Liu discloses the position estimation system according to claim 4.
However, neither Huang nor Liu, whether considered individually or in combination, explicitly disclose wherein the position estimation device further comprises an article-take-out detection unit configured to detect, based on the coordinate position of the wrist estimated by the estimation unit and on position information acquired when a predetermined article has been taken out, whether or not the article has been taken out, and an association unit configured to associate, when the article-take-out detection unit detects that the article has been taken out, the taken out article with a person whose wrist is closest to the article.
Singh teaches wherein the position estimation device further comprises an article-take-out detection unit configured to detect (¶0161, detect item puts and takes from shelves), based on the coordinate position of the wrist estimated by the estimation unit and on position information acquired when a predetermined article has been taken out (¶0162, technology disclosed uses the positions of hand joints of subjects and positions of shelves to detect proximity events. Examiner considers hand joints to indicate “wrists”), whether or not the article has been taken out (¶0161, detect item puts and takes from shelves), and an association unit configured to associate, when the article-take-out detection unit detects that the article has been taken out, the taken out article with a person whose wrist is closest to the article (¶0240, The system selects a hand (left or right) per subject per frame that has a minimum distance (of the two hands) to the hand (left or right) of another shopper or to a shelf (i.e. fixed inventory cache)… The hand associated with the proximity event).
Huang, Liu, and Singh are analogous to the claimed invention because Huang and Liu are directed towards determining a 3D position based of an object from multiple camera views and Singh is directed towards estimating human body positions. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the determination of a person taking a product of Singh into the spherical coordinates of Liu and the position determination method of Huang. The suggestion/motivation for doing so would have been to improve computational efficiency, as suggested by Singh at ¶0162, calculate the distances between hand joints of subjects and shelves per aisle or per portion of the area of real space to improve computational efficiency.
This method of improving Huang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Liu and Singh.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Huang with the teachings of Liu and Singh to obtain the invention as specified in claim 5.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Huang (US 2021/0183098) in view of Liu et al. (Liu, Shuai, et al. "3d spherical panoramic epipolar line based on essential matrix." IEEE Access 8 (2020): 192165-192176) (hereafter, “Liu”) as applied to claim 1 above, and further in view of Singh et al. (US 2022/0262069) (hereafter, “Singh”) (IDS) and Ludziejewski et al. (Ludziejewski, Jan, et al. "Integrated human tracking based on video and smartphone signal processing within the Arahub system." 2020 15th Conference on Computer Science and Information Systems (FedCSIS). IEEE, 2020) (hereafter, “Ludziejewski”).
Regarding claim 6, in which claim 1 is incorporated, Huang discloses wherein the position estimation device further comprises a skeleton estimation unit configured to estimate a skeleton of a person, as the object, imaged by each imaging device (Fig. 1C, 1D; ¶0045, identifies one or more keypoints of one or more objects detected in the respective image … keypoints K1-K14 that may be detected for a human individual. Examiner considers K1-K14 as illustrated in Figs. 1C and 1D to be the skeleton of a person), and the position estimation device comprises a second estimation unit configured to estimate coordinate positions of a neck, an elbow, and a wrist in the skeleton estimated by the skeleton estimation unit (Fig. 1C, 1D; ¶0045, the main keypoint K1 is indicated by an open circle and corresponds to the neck of the respective individual. Figs. 1C and 1D illustrate various joints including K8 which is on the wrist and K6 which is on the elbow), [a likelihood calculation unit configured to calculate likelihood, based on average distances and variance thereof from the article to the coordinate positions of the neck, the elbow, and the wrist, and a second association unit configured to associate the article with the person, based on a calculation result by the likelihood calculation unit].
However, neither Huang nor Liu, whether considered individually or in combination, explicitly disclose a likelihood calculation unit configured to calculate likelihood, based on average distances and variance thereof from the article to the coordinate positions of the neck, the elbow, and the wrist, and a second association unit configured to associate the article with the person, based on a calculation result by the likelihood calculation unit.
Singh teaches from the article to the coordinate positions of the neck, the elbow, and the wrist (¶0065, The system can also use positions of other joints such as elbow joints, or shoulder joints; ¶0099, Neck. Examiner considers the analysis of a combination of joints over each individual joint as obvious to try), and a second association unit configured to associate the article with the person (¶0240, The system selects a hand (left or right) per subject per frame that has a minimum distance (of the two hands) to the hand (left or right) of another shopper or to a shelf (i.e. fixed inventory cache)… The hand associated with the proximity event).
Huang, Liu, and Singh are analogous to the claimed invention because Huang and Liu are directed towards determining a 3D position based of an object from multiple camera views and Singh is directed towards estimating human body positions. Singh teaches the positions of the individual body parts but does not explicitly teach the combination.
It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the body positions of Singh into the spherical coordinates of Liu and the position determination method of Huang. The suggestion/motivation for doing so would have been because it would be obvious try, as there is only a finite combinations of ways to select groupings of different body locations. Further, one skilled in the art could have tried these different groupings with reasonable expectations of success.
However, none of Huang, Liu, and Singh, whether considered individually or in combination, explicitly disclose a likelihood calculation unit configured to calculate likelihood, based on average distances and variance thereof and based on a calculation result by the likelihood calculation unit.
Ludziejewski teaches a likelihood calculation unit configured to calculate likelihood, based on average distances and variance thereof (Page 111, right column, paragraphs 1-3, We define the reliability of each neighbor xj using the squared exponential kernel with a fixed length scale … the likelihood of observing a given RSSI value s for a new location x is estimated using the gaussian model with mean and variance of the closest grid point. The equations used to calculate the likelihood on page 111 depend on the mean and variance of variable wij, which is a distance metric) and based on a calculation result by the likelihood calculation unit (Page 111, right column, paragraphs 3, the likelihood. Examiner considers the combination of Ludziejewski and Singh to teach based on a calculation result as Singh teaches the association based on a specific value (distance)).
Huang, Liu, Singh, and Ludziejewski are analogous to the claimed invention because Huang and Liu are directed towards determining a 3D position based of an object from multiple camera views and Singh and Ludziejewski are directed towards estimating human body positions. It would have been obvious to a person of ordinary skill before the effective filing date of the claimed invention to incorporate the likelihood of Ludziejewski into the determination of a person taking a product of Singh, the spherical coordinates of Liu, and the position determination method of Huang. The suggestion/motivation for doing so would have been to improve accuracy, as suggested by Ludziejewski at Page 106, left column, paragraph 4, integration of multi-modal data sources for more accurate positioning and profiling may be used in smart-city and smart-home.
This method of improving Huang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Liu, Singh, and Ludziejewski.
Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date, to modify Huang with the teachings of Liu, Singh, and Ludziejewski to obtain the invention as specified in claim 6.
Allowable Subject Matter
Claim 3 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims in addition to being amended to overcome the rejection under 35 U.S.C. § 112(b).
The following is a statement of reasons for the indication of allowable subject matter: Liu discloses that in spherical coordinates, the epipolar constraint is satisfied if the two great circles containing the point projected to the object location on spheres representing the two cameras are coplanar (Fig.1; Page 192167, left column, paragraphs 2-4, Therefore, the epipolar lines p1k1 and p2k2 are respectively an arc on the great circle of two different spherical panoramic spherical surfaces). However, neither Huang nor Liu, whether considered individually or in combination, disclose the specific method of comparing p’1 to p’2 and angle β1 to β2 as recited in claim 3.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bichu et al. (US 2021/0082086) discloses using epipolar constraints in spherical coordinates for stitching images together (¶0200, The mapping between points on the first spherical surface and epipolar great circles on second spherical surface and vice versa is an epipolar constraint).
Mollis (US 2018/0342100) discloses determining the epipolar constraint in spherical coordinates (¶0061, the point in the second image lies along an epipolar great circle).
Kallakuri et al. (US 2021/0409648) disclose automated checkout systems with multiple cameras (¶0079-0080, technology disclosed is related to autonomous checkout systems or cashier-less stores … multiple sensors or cameras in the area).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIAOMAO DING whose telephone number is (571)272-7237. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XIAOMAO DING/Examiner, Art Unit 2676
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676
Prosecution Insights