DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The IDS(s) has/have been considered and placed in the application file.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1 and 13.
Claim 1 is directed to an apparatus and claim 13 to a method, and each therefore falls within a statutory category (Step 1: YES).
Under Step 2A Prong One, claim 1 recites “determines an acquisition status of acquisition target data based on a comprehensive condition and the data acquired, the comprehensive condition being a condition related to the acquisition target data” and “generates presentation information corresponding to insufficient data among pieces of the acquisition target data for which the acquisition status is inadequate.” Claim 13 recites the corresponding “determining insufficient data among pieces of acquisition target data based on the data acquired and a comprehensive condition” and “generating presentation information corresponding to the insufficient data.”
Comparing what has been collected against a stated collection requirement, judging which items fall short, and composing a message identifying what is still needed are evaluation and judgment steps that can be practically performed in the human mind or with pen and paper. A person tracking a collection list on paper performs precisely these steps. The claims therefore recite a mental process, which is an abstract idea. See MPEP 2106.04(a)(2)(III).
The generating limitation additionally falls within a second grouping. As the limitation is informed by the specification, the presentation information is an instruction directing a person to act, such as “Please turn 90 degrees to the right” (¶ 35) and “Please move 200 cm forward” (¶ 61 ), which the subject then follows so that the missing data is captured (¶ 80). Generating and presenting rules or instructions for a person to follow is a certain method of organizing human activity, specifically managing personal behavior or relationships or interactions between people. See MPEP 2106.04(a)(2)(II)(C). Claims 1 and 13 therefore recite abstract ideas under two separate groupings.
Under Step 2A Prong Two, the additional elements are the recitation of “circuitry” performing the functions and “acquires data from a sensor” ( claim 1) or “acquiring data from a sensor” (claim 13). The circuitry is recited at a high level of generality and amounts to no more than generic components applied as a tool to perform the abstract idea. See MPEP 2106.05(f). Acquiring data from a sensor is mere data gathering that is necessary antecedent activity for the recited comparison, and is therefore insignificant extra-solution activity. See MPEP 2106.05(g). The claims do not recite training or improving any model, do not recite any improvement to the sensor or to the operation of any computer, and do not recite that the presentation information is displayed, transmitted, or acted upon by any device. The judicial exception is accordingly not integrated into a practical application (Step 2A Prong Two: NO).
It is acknowledged that the specification asserts at paragraphs 12 and 81 that training data can be collected efficiently. An asserted benefit must be reflected in the claim in order to establish an improvement under MPEP 2106.05(a). The efficiency the specification describes rests on the similarity based live feedback loop of paragraphs 31 through 35 , but claims 1 and 13 recite only the outcome, that presentation information corresponding to the insufficient data is generated, and recite no mechanism by which that information is derived. A claim that recites the result without the means of achieving it does not establish a technical improvement. See Subject Matter Eligibility Example 48, claim 1. Claims 1 and 13 further recite no collection rate, no reduction in capture operations, no constraint on the sensor, and no use of the collected data. Applying generic computation to organize a data collection task, without improving the sensing or computing technology itself, is not a practical application. Recentive Analytics, Inc. v. Fox Corp., 134 F.4th 1205 (Fed. Cir. 2025). There is no particular machine, the sensor being recited generically, and no transformation.
Under Step 2B, the additional elements considered individually and as an ordered combination do not amount to significantly more. The sensor data gathering remains insignificant extras-solution activity for the reasons given above, and the generic circuitry is well-understood, routine, and conventional. This finding is supported by the specification itself, which describes the recited components only in generic terms: paragraph 19 states that “data collection processor 40 includes a processor such as a CPU” and that the recited circuitries “exist as functions of the CPU,” and paragraph 21 states that “Sensor 10 is, for example, a camera, microphone, laser, LIDAR, ultrasonic sensor, or a sensor that acquires temperature, humidity, wind speed, concentration of substances, etc.” These statements in the specification establish that the additional elements are well-understood, routine and conventional. See Berkheimer Memo and MPEP 2106.05(d)(I). Claims 1 and 13 are therefore ineligible (Step 2B: NO).
Claims 2-12 and 14-20.
The dependent claims do not cure the deficiency. Claims 2 and 14 (judging similarity), claims 3 and 15 (generating and associating metadata ), claims 4 and 16 ( setting categories and per-category target numbers and identifying the category not reached), claims 5 and 17 (selecting which shortfall to address first), and claim 11 (revising the collection requirement) each recite further evaluation or judgment steps that fall within the same mental process. Claims 6-9, 12, and 18-20 recite only what the presentation information contains or how it looks – an action indication, a video or still image, included metadata, information for sufficient data, or a differing presentation mode. These recitations describe the content of the conveyed information rather than any technical operation, and do not integrate the exception into a practical application or add significantly more. Claim 10 (audio information) recites a generic output form. Claims 2-12 and 14-20 are therefore ineligible for the same reasons.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 2, 6, 7, 14 and 16-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claims 2 and 14.
Claim 2 recites “determines whether the acquired data and the insufficient data are similar to
each other” and generating the presentation information “when the acquired data and the
insufficient data are similar to each other.” Claim 14 recites the same limitation. The “insufficient data” is defined in claims 1 and 13 as data among the acquisition target data for
which the acquisition status is inadequate, that is, data that has not been acquired. It is unclear
how a similarity is computed between data that has been acquired and data that, by definition,
does not exist in the apparatus. The specification does not resolve the ambiguity: paragraphs 31
and 32 compare the feature quantities of the acquired image data to the feature quantities of the category of the insufficient data, not to the insufficient data itself. The metes and bounds of the claim are therefore unclear.
Claims 6 and 7.
Claim 6 recites “information indicating an action for transition from the acquired data to the
insufficient data,” and claim 7 recites “a video or still image showing transition from the
acquired data to the insufficient data.” A transition from one item of data to another item of data is not an action that a subject can perform, and it is unclear what is being transitioned.
Paragraphs 97 and 102 of the specification describe information indicating actions for transition
from the current state to the state of the insufficient data, which is not what the claims recite. The scope of claims 6 and 7 is therefore unclear.
Claim 16.
Claim 16 recites “a number of pieces of the data to be acquired for each category of the data”
and “determining a category of data for which the number of pieces of data is insufficient.”
Claim 13 introduces both “data” acquired from the sensor and “acquisition target data,” and it is unclear which of the two “the data” refers to. The claim is further inconsistent in reciting “a
number of pieces of the data” and then “the number of pieces of data,” which lacks antecedent
basis in the form recited.
Claims 17-19.
Claim 13 establishes “insufficient data.” Claim 17 recites “insufficient pieces of the data,” and
claims 18 and 19 each recite “an insufficient piece of the data.” These terms lack antecedent
basis and it is unclear whether they are intended to refer to the “insufficient data” of claim 13 or to introduce a different quantity. Claim 17 additionally recites “the generating the presentation information involves generating the presentation information from the data being the prioritized acquisition target,” which is grammatically unclear.
Claim 18.
Claim 18 further recites “information that indicates a difference in an action for transition from a current state to a state of an insufficient piece of the data.” It is unclear what the recited “difference” is measured between. Paragraph 97 describes information indicating actions, characterized as state differences, for transition from the current state to the state of the
insufficient data; a difference in an action has no counterpart in the specification. The scope of
claim 18 is therefore unclear.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-6, 10 and 13-18 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Turkelson et al., US 2020/0210768 A1 (hereinafter “TURKELSON”).
Claim 1.
TURKELSON discloses a presentation information generation apparatus, comprising:
a data acquiring circuitry which, in operation, acquires data from a sensor (TURKELSON: “Some embodiments may generate a user interface that guides users to acquire particularly robust training data for object recognition models and other computer vision algorithms. Some embodiments may cause mobile computing devices . .. to present such a user interface, and a camera of the mobile computing device may obtain videos or other sets of images of an object. The images may be added to a training data set used to train a model” (¶ 25). This teaches acquisition of data from a camera sensor.);
a determining circuitry which, in operation, determines an acquisition status of acquisition target data based on a comprehensive condition and the data acquired, the comprehensive condition being a condition related to the acquisition target data (TURKELSON: “At step 306, a determination may be made as to whether the number of images determined at step 304 is less than a threshold value . ... the threshold value may be 1 or more images depicting the object from A different perspectives, 1 or more images depicting the object with B different ambient light levels, 1 or more images depicting the object with C different backgrounds, 1 or more images depicting the object having D different configurations” (¶ 93). This teaches determining acquisition status against a condition set for the acquisition target data.); and
an information generating circuitry which, in operation, generates presentation information corresponding to insufficient data among pieces of the acquisition target data for which the acquisition status is inadequate (TURKELSON: “At step 310, criteria missing from the images of the object (e.g., from the training data set) may be identified”(¶ 97); “At step 312, an image capture task may be generated, where the image-capture task includes instructions describing how a video of the object is to be captured to obtain images of the object including the missing criteria” (¶ 98). This teaches generating presentation information corresponding to the identified insufficient data.).
Claim 2.
TURKELSON discloses further comprising: a similarity calculating circuitry which, in operation, determines whether the acquired data and the insufficient data are similar to each other, wherein the information generating circuitry generates the presentation information when the acquired data and the insufficient data are similar to each other (TURKELSON: “within specified parameters, like within tolerances for object distance, camera pose, lighting intensity, etc.). For instance, some embodiments may infer camera pose with a Kalman filter and compare an inferred pose to a target pose to determine an error vector (e.g., in three or six spatial dimensions). Some embodiments may cause the mobile computing device to present an overlaid augmented reality display on a user interface depicting captured images or video based on the error vector, like arrows indicating how the camera should be manipulated in space to reduce the error vector” (¶ 31). This teaches comparing the currently acquired pose against the target pose of the lacking image and generating the guidance from that comparison.).
Claim 3.
TURKELSON discloses the presentation information generation apparatus according to claim 1, wherein the data acquiring circuitry generates metadata that includes information on a feature quantity of the acquired data and associates the metadata with the acquired data (TURKELSON: “the video may be segmented into a plurality of frames, where each frame may include information indicating the camera's pose (e.g., with six coordinates, three indicating spatial position and three indicating orientation) with respect to the object, ambient lighting conditions, eye gaze information” (¶ 34). This teaches per-frame metadata of feature quantities associated with the acquired image.).
Claim 4.
TURKELSON discloses the presentation information generation apparatus according to claim 1, wherein: the comprehensive condition includes a category for classifying the acquisition target data and an acquisition target number for each category, and the determining circuitry determines, as the insufficient data, a category for which the acquisition target number is not reached (TURKELSON: “the training data set may be determined to include a satisfactory number of images if the training data set includes a first number of images depicting the object from A or more different perspectives, a second number of images depicting the object with B different ambient light levels, a third number of images depicting the object with C different backgrounds, a fourth number of images depicting the object having D different configurations” (¶ 96); “at least 100 images depict the drill from a different perspective (e.g., 0-degrees with respect to a front face of the object, 180-degrees with respect to the front face of the object, etc.)” (¶ 96); “at step 308, the dimensions with which a number of images do not satisfy the criteria may be identified” (¶ 97). This teaches per-category target numbers and identification of the category falling short.).
Claim 5.
TURKELSON discloses the presentation information generation apparatus according to claim 1, wherein: the determining circuitry determines prioritized data, which is a target for prioritized acquisition, from among pieces of the insufficient data, and the information generating circuitry preferentially generates the presentation information for the prioritized data (TURKELSON: “Some embodiments may select image-capture tasks for users based on a marginal entropy gain of the image-capture task” (¶ 26); “if only a single image of a jacket (e.g., in a particular color, pattern, or size; draped, worn, or laying on a surface) is included within a fashion-related corpus of training images, then (e.g., in response) an image-capture task may be generated (e.g., selected, composed, or otherwise obtained) for obtaining additional images of the jacket” (¶ 26). This teaches selecting which shortfall to address first and generating the instruction for that selection.).
Claim 6.
TURKELSON discloses the presentation information generation apparatus according to claim 1, wherein the information generating circuitry generates, as the presentation information, information indicating an action for transition from the acquired data to the insufficient data (TURKELSON: “Some embodiments may include generating visual indicators instructing the user operating mobile computing device 104 as to the actions that he/she is to perform. For example, if mobile computing device 104 is to be moved to the right, user interface generation subsystem 118 may generate a visual indicator (e.g., an arrow) pointing in a direction of the user's right” (¶ 81). This teaches an indication of the action required to reach the lacking image.).
Claim 10.
TURKELSON discloses the presentation information generation apparatus according to claim 1, wherein the information generating circuitry generates, as the presentation information, information including audio information (TURKELSON: “may provide feedback (e.g., visual, audible, haptic, etc.) via the user interface or via other components of the mobile computing device indicative of the feedback” (¶ 25). This teaches audible presentation information.).
Claims 13-18.
Claims 13-18 recite a presentation information generation method reciting limitations that mirror those of claims 1-6, and are rejected on the same grounds and the same citations, mutatis mutandis. Claim 13 corresponds to claim 1, claim 14 to claim 2, claim 15 to claim 3, claim 16 to claim 4, claim 17 to claim 5, and claim 18 to claim 6. TURKELSON discloses the method as a process performed by the disclosed system (TURKELSON: “the sub-tasks may cause mobile computing device 104 to be adjusted in a manner (e.g., rotated about an axis, moved in a certain direction with respect to the object, etc.) to allow the video to capture the object from a perspective, lighting condition, background setting, configuration, or a combination thereof, which currently is not included within the training data for training an object recognition model to recognize that object” (¶ 80). This teaches the recited method
steps.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 7-9, 12, 19 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over TURKELSON in view of Shreve et al., US 2023/0403459 A1 (hereinafter “SHREVE”).
Claim 7 and 19.
The presentation information generation apparatus according to claim 1, wherein: the acquired data is an image, and the information generating circuitry generates, as the presentation information, (TURKELSON: “a visual indicator (e.g., an arrow) pointing in a direction of the user's right” (¶ 81). This teaches a static indicator but not a video showing the transition.).
TURKELSON does not specifically teach “a video or still image showing transition from the acquired data to the insufficient data.” However, SHREVE, in the same field of endeavor of
guiding a user to capture images a training dataset lacks, teaches animated guidance showing the required transition (SHREVE: "Module 216 can use the statistical measures estimated by
module 214 and guide the user to capture additional images to ensure obtaining all possible
views of each object under all lighting conditions with and without occlusions and without
excessive blur … Module 216 can provide this guidance as instructions (similar to
instructions 176/198 in FIG. 1) to the user using AR features, such as arrows, waypoints,
animations (e.g., to tilt the screen up/down, to move closer/further, etc.)”(¶ 67). This
teaches an animation showing the transition to the lacking view.).
Therefore, it would have been obvious to one of ordinary skill in the art to combine TURKELSON and SHREVE before the effective filing date of the claimed invention. The motivation for this combination would have been that TURKELSON signals a needed capture with a single static arrow, which tells the user the immediate direction to move but not which
views remain outstanding or how many; SHREVE's animated waypoints and color-coded coverage heatmap resolve that ambiguity by showing the full transition path and distinguishing views already captured from views still required, thereby eliminating redundant re-capture of
already covered views and reducing the number of capture passes needed to complete the
dataset. SHREVE supplies the reason itself, stating that “current techniques do not ensure quality of coverage or viewing conditions, nor do they provide quality estimates of individual images” (¶ 36). This rationale corresponds to MPEP 2143(A), combining prior art elements according to known methods to yield predictable results, and to MPEP 2143(G), teaching, suggestion or motivation in the art. A reasonable expectation of success is present because both references render capture guidance on the display of the same class of camera equipped device from the same per-view coverage bookkeeping, so substituting or adding SHREVE's visualization requires no change to how TURKELSON identifies the missing criteria.
Claim 8 and 20.
TURKELSON and SHREVE teach wherein the information generating circuitry generates, as the presentation information, information including the metadata (SHREVE: "Module 216 can also generate a final report (not shown) which lists the status of each object as complete or not … This final report can also indicate a 'percentage complete' bar, including a number of images remaining which need to be captured to reach completeness, as well as the various corresponding scene characteristics, such as views (distance and angle), lighting conditions, occlusion conditions, and blur conditions” (¶ 75). This teaches presentation information that includes the metadata characterizing the captured images.).
Claim 9.
TURKELSON and SHREVE teach wherein the information generating circuitry further generates the presentation information corresponding to sufficient data among the pieces of acquisition target data for which the acquisition status is adequate (SHREVE: “A heatmap green color 528 can indicate that the user has already captured images from that view” (¶ 70); “By viewing table 600, the user can quickly determine that the tissue box has not yet been covered at all (i.e., no images have been captured at all), while both the salt shaker and the pepper shaker may need more coverage (i.e., additional images are required)” (¶ 73). This teaches presentation information for the data whose acquisition status is adequate alongside that which is inadequate.).
Claim 12.
TURKELSON and SHREVE teach wherein the information generating circuitry generates, as the presentation information, information for which a presentation mode differs depending on a content to be presented (SHREVE: “instructions/user guidance 522 can indicate the heatmap as a geodesic polyhedron, with various colors on one or more face representing different statuses of coverage of the image. For example, a heatmap red color 524 can indicate that images have not yet been captured from that view and still need to be captured. A heatmap yellow color 526 can indicate that the user is currently looking at the object from that view, but that images have not yet been captured from that view. A heatmap green color 528 can indicate that the user has already captured images from that view” (¶ 70). This teaches a
presentation mode that differs according to the content presented.).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over TURKELSON in view of Abdi Taghi Abad et al., US 10,534,984 B1 (hereinafter “ABAD”).
Claim 11.
TURKELSON discloses the presentation information generation apparatus according to claim 1, further comprising:
TURKELSON does not teach “a controlling circuitry which, in operation, changes the comprehensive condition based on the acquired data.” However, ABAD, in the same field of endeavor of assembling training data by category, teaches revising the per-category collection target from the data already collected (ABAD: “one or more training batches (or training sets) may be generated by at least one computing device, where each training batch has a constant size and contains a set number of samples that are associated with each class” (C2:L 7-11); “Based on the accuracy of each class, the one or more training batches may be adjusted or modified for use in the next training session. For instance, the number of samples associated with the class exhibiting low accuracy may be increased in the training batch to improve the accuracy of that class" (C2:L18-24); "The adjustment may be dynamic" (C2:L29). This teaches changing the per-category collection condition based on the data already acquired.).
Regarding the requirement that the change be made “based on the acquired data,” the per-class accuracy on which ABAD conditions the change is itself computed by training on the samples already collected (ABAD: “The one or more training batches may then be used to train the classification model for a number of iterations, e.g., epochs. Based on the training, an accuracy corresponding to each class may be determined” (C2:L12-15).). The revised sample counts are therefore a function of the acquired data, as claimed.
ABAD is analogous art. It is from the same field of endeavor as the claimed invention and as TURKELSON, namely assembling a per-category training data set for a classification model, and it is reasonably pertinent to the particular problem with which the inventor was concerned, namely how many samples of each category should be required. See MPEP 2141.01(a).
Therefore, it would have been obvious to one of ordinary skill in the art to combine TURKELSON and ABAD before the effective filing date of the claimed invention. The motivation for this combination would have been that TURKELSON fixes its per-dimension image counts in advance, so a category whose weakness only becomes apparent after collection begins continues to be collected against a stale target; ABAD adjusts the per-class sample counts from the results obtained on the data already collected, so that categories which remain weak receive additional samples. Applying ABAD's adjustment to TURKELSON's per-category thresholds allows the collection targets to track the actual state of the dataset, avoiding both overcollection of well-covered categories and under-collection of weak ones. This rationale corresponds to MPEP 2143(A), combining prior art elements according to known methods to yield predictable results. A reasonable expectation of success is present because both references express the collection requirement as a count of samples per category, so ABAD's adjustment operates on the same quantity TURKELSON already maintains and requires no change to TURKELSON's capture or identification steps.
Conclusion
The following prior art is made of record but not relied upon is considered pertinent to the applicant’s disclosure. Ueta et al. (US 11,579,904 B2) discloses a learning data collection device that determines a captured image is unsuitable as training data and performs a notification operation prompting the image capturing person to reshoot. Soda et al. (US 2020/0118020 A1), the English-language family member of the Japanese publication cited by applicant as PTL 4, discloses collecting vehicle data under user-specified collection conditions and providing the user with information regarding the collection period required to reach a desired collection amount. See the attached PTO-892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674
Prosecution Insights