DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of priority from Foreign Application No. JP2022-123106, filed August 2, 2022 and PCT Application No. PCT/JP2023/026535, filed July 20, 2023.
Information Disclosure Statement
The information disclosure statement (“IDS”) filed on January 17, 2025 was reviewed and the listed references were noted.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, AND RECORDING MEDIUM FOR ACQUIRING AN IMAGE SUITABLE FOR A USE CASE OF AI.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “selection unit that selects…”, “display control unit that displays…”, input means…”, “process processing unit…”, and “output unit that outputs…” in claims 1-17.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because “a computer-readable recording medium recording a program…” could refer to transitory forms of signal transmission (i.e., “signals per se”) which does not fall under any of the statutory categories (see MPEP 2106.03(I)).
Claim interpretation affects the evaluation of both criteria for eligibility. For example, in Mentor Graphics v. EVE-USA, Inc., 851 F.3d 1275, 112 USPQ2d 1120 (Fed. Cir. 2017), claim interpretation was crucial to the court’s determination that claims to a "machine-readable medium" were not to a statutory category. In Mentor Graphics, the court interpreted the claims in light of the specification, which expressly defined the medium as encompassing "any data storage device" including random-access memory and carrier waves. Although random-access memory and magnetic tape are statutory media, carrier waves are not because they are signals similar to the transitory, propagating signals held to be non-statutory in Nuijten. 851 F.3d at 1294, 112 USPQ2d at 1133 (citing In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007)). Accordingly, because the BRI of the claims covered both subject matter that falls within a statutory category (the random-access memory), as well as subject matter that does not (the carrier waves), the claims as a whole were not to a statutory category and thus failed the first criterion for eligibility.
The rejection of claim 19 may be overcome by amending the claim to, for example, recite as: “a non-transitory computer-readable recording medium recording a program…”.
Claims 1, 11-14 and 18-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a system, method, and computer-readable medium for acquiring learning images suitable for a use case of a learning model. Consider method claim 18:
Step 1:
With regard to Step 1, the instant claim is directed to a method or a process; and therefore, the claim is directed to one of the statutory categories of invention.
Step 2A, Prong One:
With regard to 2A, Prong One, the limitation “selecting a learning image used for learning of a learning model, according to a use case of the learning model using an image as an input, from among an image group held in advance” as drafted, recite an abstract idea, such as a process that, under its broadest reasonable interpretation, covers performance of the limitations manually and in the mind of a person. That is, a user or person skilled in the art may select an image from a group of images that would be best suited to train a learning model based on what the use case of the model is (e.g., selecting a cat image when the use case is identifying cats). This is the concept that falls under the grouping of abstract ideas mental processes, i.e., a concept performed in the human mind, evaluation, judgement, and/or opinion of the user.
Step 2A, Prong Two:
The 2019 PEG defines the phrase “integration into a practical application” to require an additional step or a combination of additional steps in the claim to apply, rely on, or use the judicial exception. In addition, with respect to the computer-readable medium claim of claim 19, the mere recitation of a generic storage medium to store programming instructions of the recited/identified abstract idea does not integrate the identified abstract idea into a practical application. Accordingly, the above-mentioned additional elements/limitations do not integrate the abstract idea into a practical application; and therefore, the independent claims recite an abstract idea.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B. The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception, because as discussed above with respect to integration of the abstract idea into practical application, the additional elements/limitations to perform the recited steps, amount to no more than insignificant extra-solution activity. Mere instructions to apply an exception using a generic component cannot provide an inventive concept. Therefore, independent claims 1, 18 and 19 are not patent eligible. In addition, claims 11-14 of the instant application provide limitations that both individually or in combination do not integrate the identified abstract idea into a practical application or provide significantly more than the identified abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 6-12 and 15-19 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Gil Elbaz (US 2023/0185440 A1).
Regarding claim 1, Elbaz teaches an information processing device comprising:
a selection unit that selects a learning image used for learning of a learning model (Para. [0038], data uploader may automatically select portions of uploaded images based on the training subject for a machine learning model), according to a use case of the learning model using an image as an input (Para. [0038], if an ML model needs to be trained to identify cats (i.e., use case of the learning model), data uploader may crop cats from uploaded images), from among an image group held in advance (Para. [0038], data uploader alone or with the GUI tool may automatically select portions of uploaded images (i.e., image group held in advance) to use as an initial dataset to generate a synthetic dataset).
Regarding claim 2, Elbaz teaches the information processing device according to claim 1, further comprising:
a display control unit that displays input means for a user to input the use case (Para. [0153], the GUI (i.e., display control unit) may include a target object type selector configured to receive a target object type input from the user indicative of at least one selected target object type (i.e., use case) to feature in the synthetic dataset).
Regarding claim 3, Elbaz teaches the information processing device according to claim 2, wherein
the input means to input the use case includes any one of a pull-down menu, a text box, a combo box, and an icon (Para. [0153], the target object type selector may include a menu, a drop-drop list (i.e., pull-down menu), a set of options with corresponding selector elements such as checkboxes or radio buttons, or other user interface elements to permit the user to select one or more options).
Regarding claim 4, Elbaz teaches the information processing device according to claim 2, further comprising:
a process processing unit that executes process processing based on information regarding a camera that captures an image to be input into the learning model, on the learning image (Para. [0076], System may use camera characteristics (i.e., information regarding a camera) using extracted camera parameter values to generate data (e.g., images) that match the extracted camera parameter values to include in a synthetic dataset (i.e., images to be input into the learning model)).
Regarding claim 6, Elbaz teaches the information processing device according to claim 4, wherein
the display control unit displays a list of images selected as the learning image, before the process processing is executed on the learning image (Para. [0124], control may be configured to display a sample image to illustrate to the user what the different options may look like for the selected control parameter allowing the user to see a preview of sample synthetic images to be generated based on a current set of user selections allows the user to determine (before the entire synthetic dataset is generated) whether the synthetic dataset will exhibit the user's desired characteristics).
Regarding claim 7, Elbaz teaches the information processing device according to claim 4, wherein
the display control unit displays an image on which the process processing is executed, before the process processing is executed on the learning image (Para. [0124], control may be configured to display a sample image to illustrate to the user what the different options may look like for the selected control parameter allowing the user to see a preview of sample synthetic images to be generated based on a current set of user selections allows the user to determine (before the entire synthetic dataset is generated) whether the synthetic dataset will exhibit the user's desired characteristics).
Regarding claim 8, Elbaz teaches the information processing device according to claim 4, wherein
the display control unit displays input means to input the information regarding the camera (Para. [0140], GUI tool with additional controls defining variations in camera characteristics for use in generation of synthetic datasets, according to some embodiments of the present disclosure. The values of the camera intrinsic parameters may be input via a user interface element such as a drop-down list, a text box, a text box with a corresponding selector, or other user interface element that permits the user to select a discrete value).
Regarding claim 9, Elbaz teaches the information processing device according to claim 8, wherein
the information regarding the camera includes information regarding at least one of an image sensor and a lens provided in the camera (Para. [0029], camera specifications may be input via the tool and used as a basis for generating the synthetic image data. For example, a user may specify any one or more of a camera orientation, resolution, field of view (FOV), aspect ratio, camera sensor size, camera sensor type, wavelength sensitivity range, etc).
Regarding claim 10, Elbaz teaches the information processing device according to claim 9, wherein
the input means to input the information regarding the camera includes input means to input at least any one of a model or characteristics of the image sensor and a type of the lens (Para. [0029], camera specifications may be input via the tool and used as a basis for generating the synthetic image data. For example, a user may specify any one or more of a camera orientation, resolution, field of view (FOV), aspect ratio, camera sensor size, camera sensor type, wavelength sensitivity range, etc).
Regarding claim 11, Elbaz teaches the information processing device according to claim 1, wherein
the selection unit selects the learning image, according to at least any one of, input by the user, a type of a subject, a type of a background, brightness, a frequency, and a contrast, from among the image group (Para. [0153], the GUI may include a target object type selector configured to receive a target object type input from the user indicative of at least one selected target object type to feature in the synthetic dataset).
Regarding claim 12, Elbaz teaches the information processing device according to claim 1, wherein
the selection unit adds an image selected from among the image group based on an image input by a user or the image input by the user, as the learning image (Para. [0082], sample image upload interface may be configured to help select input data (for example, static images and motion images) to upload using data uploader and store as images. Sample image upload interface may be presented as a screen within GUI tool, a window overlaying GUI tool, or an independent window. Sample image upload interface may include a drag-and-drop window or other user interface element to select sample images from a file listing from a directory stored on a local computing device (e.g., computing device) displaying GUI tool).
Regarding claim 15, Elbaz teaches the information processing device according to claim 1, further comprising:
an output unit that outputs the learning image to a learning device that learns the learning model (Para. [0034], a machine learning (ML) model manager 120 configured to train machine learning models using generated datasets); and
a display control unit that displays a list of the learning images, before the learning image is output (Para. [0088], Preview interface 340 may be configured to aid in previewing synthetic datasets (e.g., training datasets 152) generated by system 100. Preview interface 340 may randomly select an image in images 210 of synthetic dataset 200 for a user of GUI tool 300 to preview. Preview interface 340 may be configured to provide preview images 210 of synthetic dataset).
Regarding claim 16, Elbaz teaches the information processing device according to claim 15, wherein
the display control unit displays a list of at least one of metadata and a statistical amount corresponding to the learning image, before the learning image is output (Para. [0146], Preview pane 540 may include a preview image 1241 to assist the user in visualizing the changes made to a sample image by adjusting parameter values in controls 1231 and 1232. Para. [0149], Values of camera parameters of controls 1131-1135 and 1231-1232 may be stored as camera characteristics 167. System 100 may automatically generate simulated camera parameters and their default values from sample images 141 uploaded using sample image upload interface 310. In some embodiments, meta properties may include lighting parameters, such as type of light source, brightness level, lighting color, lighting direction, and a number of light sources).
Regarding claim 17, Elbaz teaches the information processing device according to claim 15, wherein
the display control unit displays at least any one of a statistical amount of a data set including a plurality of the learning images, information indicating a type of a subject or a background of each of the plurality of learning images, and information indicating a distribution of the type of the subject or the background in the data set, before the learning image is output (Para. [0146], Preview pane 540 may include a preview image 1241 to assist the user in visualizing the changes made to a sample image by adjusting parameter values in controls 1231 and 1232. Para. [0149], Values of camera parameters of controls 1131-1135 and 1231-1232 may be stored as camera characteristics 167. System 100 may automatically generate simulated camera parameters and their default values from sample images 141 uploaded using sample image upload interface 310. In some embodiments, meta properties may include lighting parameters, such as type of light source, brightness level, lighting color, lighting direction, and a number of light sources).
Claim 18 recites a method with steps corresponding to the elements of the system recited in Claim 1. Therefore, the recited steps of this claim are mapped to the proposed reference in the same manner as the corresponding elements in its corresponding system claim.
Claim 19 recites a computer-readable storage medium storing a program with instructions corresponding to the elements recited in Claim 1. Therefore, the recited programming instructions of this claim are mapped to the proposed reference in the same manner as the corresponding steps in its corresponding device claim. Additionally, the Elbaz reference discloses a computer readable storage medium (Para. [0151], a non-transitory, computer-readable medium).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Gil Elbaz (US 2023/0185440 A1) in view of Fergus Kidd (US 2022/0262008 A1).
Regarding claim 5, Elbaz teaches the information processing device according to claim 4, as described above.
Although Elbaz teaches using camera parameter values to generate a learning image (Elbaz, Para. [0076]), Elbaz does not explicitly teach “the process processing unit executes the process processing by adding at least one of deterioration and noise generated in an image by imaging by the camera to the learning image”. However, in an analogous field of endeavor, Kidd teaches the image generation system may add impulse noise, fat-tail distributed noise, and/or Gaussian noise, among other examples, to the image of the object of interest and/or the selected background image to simulate noise that would be generated by a camera in a real world scenario (Kidd, Para. [0055]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the device of Elbaz with the teachings of Kidd by including adding noise to the learning image that would be generated by a camera in a real world scenario. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for generating images for model training, as recognized by Kidd. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Gil Elbaz (US 2023/0185440 A1) in view of Pooja Rangarajan (US 2022/0406066 A1).
Regarding claim 13, Elbaz teaches the information processing device according to claim 1, as described above.
Although Elbaz teaches using a user input image as the learning image (Para. [0082]), Elbaz does not explicitly teach “the selection unit adds an image generated on the basis of a CG model input by the user, as the learning image”. However, in an analogous field of endeavor, Rangarajan teaches the training data generation unit is configured to use the model of the object of interest received based on user input data to generate the synthetic training data set (Rangarajan, Para. [0147]). The set of training data, (e.g., training images I), may be generated based on at least one 3D model (10a-c) of an object of interest by varying and/or amending at least one (in particular pre-set) parameter being characteristic for a weather condition, a number of vehicles, a number of people, a timing of the day, an orientation and/or a rotation (Rangarajan, Para. [0174]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date to modify the device of Elbaz with the teaching of Rangarajan by including adding an image generated on the basis of a CG model (i.e., 3D model) as the learning image (i.e., training data). One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for generating a synthetic training data set for training a machine learning computer vision model, as recognized by Rangarajan. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Gil Elbaz (US 2023/0185440 A1) in view of Tomoharu Kiyuna (US 2024/0203100 A1).
Regarding claim 14, Elbaz teaches the information processing device according to claim 1, as described above.
Although Elbaz teaches a table showing values of ranges of characteristic variations to include in generate synthetic datasets (Elbaz, Para. [0127]), Elbaz does not explicitly teach “the selection unit selects the learning image, based on a table in which a degree at which each image included in the image group is suitable for learning the learning model used for a predetermined use case is registered”. However, in an analogous field of endeavor, Kiyuna teaches a prediction model for predicting a probability that a predetermined feature is included in the selected training partial images (step S33). Next, the prediction means performs prediction for all the partial images using the trained prediction model (step S34). Next, the partial image selection means selects the plurality of training partial images to be used in a next training based on predicted values for all the partial images (Kiyuna, Para. [0083]).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the device of Elbaz with the teachings of Kiyuna by including predicting values for each image based on the probability of the feature being included in the image (i.e., suitable for learning the learning model for a predetermined use case) and using these values to select learning images. One having ordinary skill in the art would have been motivated to combine these references because doing so would allow for selecting an image for training that has the highest predicted value, as recognized by Kiyuna. Thus, the claimed invention would have been obvious to one having ordinary skill in the art before the effective filing date.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emma Rose Goebel whose telephone number is (703)756-5582. The examiner can normally be reached Monday - Friday 7:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Emma Rose Goebel/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights