Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendments and remarks submitted 05/14/2026 have been entered and considered, Claims 1, 3-7 are amended. Claim 2 is cancelled. This action is made final.
Response to Arguments
Applicant’s arguments filed on 05/14/2026 have been fully considered but are moot because they don’t apply to the reference(s)/combination(s) in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-7 are rejected under 35 U.S.C. 103 as being unpatentable over Case et al (US20230290110) in view of Kim et al (US20220207294).
Regarding Claim 1. Case teaches A non-transitory storage medium storing a supervised data generation program for generating supervised data to generate a trained model for outputting a result from identifying a target in response to input of image data of an image including the target (Case, abstract, the invention describes systems and methods for generating synthetic training data for machine learning models. Images of a particular object (such as an aircraft) can be received and processed to cutout the object (i.e., separate the object from the background) from the received image. The systems and methods described herein can detect areas in the background images to place an object. Once a suitable area has been detected, the cutout object image can be superimposed on the background image at the location determined to be suitable for placing the object. Superimposing the object onto the background image can include blending the two images using a plurality of blending techniques to reduce artifacts that may bias a supervised training process.
[0108] FIG. 10 illustrates an example of a computing system 1000, in accordance with some examples of the disclosure. System 1000 can be a client or a server. As shown in FIG. 10. system 1000 can be any suitable type of processor-based system, such as a personal computer, workstation, server, handheld computing device (portable electronic device) such as a phone or tablet, or dedicated device. The system 1000 can include, for example, one or more of input device 1020, output device 1030, one or more processors 1010, storage 1040, and communication device 1060. Input device 1020 and output device 1030 can generally correspond to those described above and can either be connectable or integrated with the computer.
[0110] Storage 1040 can be any suitable device that provides storage, such as an electrical, magnetic, or optical memory including a RAM, cache, hard drive, removable storage disk, or other non-transitory computer readable medium.), the program causing a computer to perform operations comprising:
selecting one or more first target images randomly or in a predetermined order from an image group including a plurality of types of target images corresponding to targets to be identified by the trained model; (Case, [0007] Presented herein are systems and methods for generating synthetic training data for machine learning models according to examples of the disclosure. In one or more examples, images of a particular object (such as an aircraft) can be received and processed so as to cutout the object (i.e., separate the object from the background) from the received image. In the example of when aircrafts are the object of interest, the images can come from a variety of sources such as existing satellite imagery, CAD models, and images of scale models of aircraft.).
Case fails to explicitly teach, however, Kim teaches generating a background image by performing a transformation process (Kim, abstract, the invention describes a method for augmenting training data by combining an object and a background with each other. The method includes extracting an object image, wherein the object image is a machine learning target; determining a type of the object image; receiving a background image, wherein the background image comprises a plurality of different background regions; identifying a first background region and a second background region among the plurality of different background regions; and combining the object image with the first background region and the second background region to augment training data, wherein combining the object image with the first background region and the second background region includes randomly positioning an image of a first type object corresponding to the first background region into the first background region, and randomly positioning an image of a second type object corresponding to the second background region into the second background region.
[0060] After the available object-positioned region is specified in the background image, the object and background combination process is completed by randomly positioning the corresponding object into the specified object-positioned region (S250). The positioning of the object may include randomly positioning the object in the corresponding region regardless of a location and number of objects. The device performs labeling while randomly positioning the corresponding object in the specified object positioned region. In other words, the device stores size information and location information about the object via the labeling, so that when a machine learning program analyzes the image, the program may learn which object is present at which location. In this connection, the category of the object may also be labeled. Codec and other state information may also be labeled. When the object is combined with the background, the object may be inverted, may be rendered in a black and white manner, and may be rotated, or may be subjected to scaling, flipping, perspective transforming, and lighting conditioning. Information on the above processing may be recorded in a label.
Case, [0016] Optionally, determining a placement area of the first background image to place the first cutout object image upon comprises generating a half-tone representation of the first background image.).
Case and Kim are analogous art because they both teach method of generating synthetic training data for machine learning models. Kim further teach storing the location information of the second target object and position it to the background image. Therefore, it would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention, to modify the synthetic training image generation method (taught in Case), to further use the location data to position the target object to the background image (taught in Kim), so as to generation synthetic training image with consistency and improved performance of machine learning (Kim, [0010]).
The combination of Case and Kim further teaches generating a background image by performing a transformation process on transformed images obtained by dividing the selected first target images to obtain transformed images, and arranging the transformed images randomly or in a predetermined order until a background frame is filled or filled by a predetermined amount; (Case, [0011] In one or more examples, a method for generating synthetic training images configured to train a machine learning classifier to detect the presence of one or more objects in an image comprises: receiving a plurality of object images, wherein each object image of the plurality of object images comprises an object to be identified by the machine learning classifier, segmenting the image of the object from each object image of the plurality of object images to generate a plurality of cutout object images, receiving a plurality of background images, mapping a first cutout object image of the plurality of cutout object images and a first background image of the plurality of background images to a common color map, determining a placement area of the first background image to place the first cutout object image upon, superimposing the first cutout object image onto the determined placement area of the first background image to generate a first synthetic training, and annotating the generated first synthetic training image with a location of the superimposed first cutout object image on the generated synthetic training image.).
selecting a second target image from the image group and generating a supervised image by combining the second target image with the background image (Kim, [0067] FIG. 4 is an exemplary diagram showing an image of training data augmented by combining an object with a background image according to the method in FIG. 3.
[0068] Referring to an upper drawing of FIG. 4, the device may divide a background image containing a sidewalk, a road, a river, and buildings into a plurality of regions and thus define available object-positioned regions, based on the division result. In an embodiment of FIG. 4, a region 410 may be defined as a sidewalk region on which a person travels, and a region 420 may be defined as a road region on which a vehicle travels, and other regions may be defined as the object-excluded region.
[0070] Referring to a lower drawing of FIG. 4, the device positions only person objects (412-1, 412-2) in the region 410, and positions only vehicle objects (not shown) in the region 420. In this manner, the augmented training data may be generated by combining the object and the background with each other more realistically.).
generating, as a set of the supervised data, the supervised image and positional information indicating a position of the second target image with respect to the supervised image. (Kim, [0059] … According to the determination result, the device may divide the input background image into one or more regions to specify a region in which the object is to be positioned. The object background matching table defines an image feature of the background region matching with the object using a plurality of parameters related to the image.
[0060] … In other words, the device stores size information and location information about the object via the labeling, so that when a machine learning program analyzes the image, the program may learn which object is present at which location. In this connection, the category of the object may also be labeled. Codec and other state information may also be labeled. When the object is combined with the background, the object may be inverted, may be rendered in a black and white manner, and may be rotated, or may be subjected to scaling, flipping, perspective transforming, and lighting conditioning. Information on the above processing may be recorded in a label.)
Claim 3 is similar in scope as Claim 1, and thus is rejected under same rationale.
Claim 4 is similar in scope as Claim 1, and thus is rejected under same rationale.
Claim 5 is similar in scope as Claim 1, and thus is rejected under same rationale. Claim 5 further requires:
A training apparatus comprising one or more processors (Case, [0108] FIG. 10 illustrates an example of a computing system 1000, in accordance with some examples of the disclosure. System 1000 can be a client or a server. As shown in FIG. 10. system 1000 can be any suitable type of processor-based system, such as a personal computer, workstation, server, handheld computing device (portable electronic device) such as a phone or tablet, or dedicated device. The system 1000 can include, for example, one or more of input device 1020, output device 1030, one or more processors 1010, storage 1040, and communication device 1060.
[0111] Processor(s) 1010 can be any suitable processor or combination of processors, including any of, or any combination of, a central processing unit (CPU), field programmable gate array (FPGA), and application-specific integrated circuit (ASIC). Software 1050, which can be stored in storage 1040 and executed by one or more processors 1010, can include, for example, the programming that embodies the functionality or portions of the functionality of the present disclosure (e.g., as embodied in the devices as described above).)
configured to generate, with the plurality of supervised data pieces, a trained model for outputting a result from identifying a target in response to input of image data of an image including the target (Case, [0070] A boneyard satellite image, such as the image 100 with annotations 102 can be used as part of a supervised training process configured to train a classifier to automatically detect the presence of aircraft in a satellite image. FIG. 2 illustrates an exemplary supervised training process for generating a machine learning model according to examples of the disclosure. In the example of FIG. 2, the process 200 can begin at step 202 wherein a particular characteristic for a given binary machine learning classifier is selected or determined (such as the presence of an aircraft, aircraft type, orientation of an aircraft, etc.). In one or more examples, step 402 can be optional, as the selection of characteristics needed for the machine learning classifiers can be selected beforehand in a separate process.
[0073] In one or more examples, and in the case of segmentation or region-based classifiers such as region based convolutional neural networks (R-CNNs), the training images can be annotated on a pixel-by-pixel or regional basis to identify the specific pixels or regions of an image that contain specific characteristics. For instance in the case of R-CNNs, the annotations can take the form of bounding boxes or segmentations of the training images.).
Regarding Claim 6. The combination of Case and Kim further teaches The non-transitory storage medium according to claim 1, wherein the supervised image includes as a transformed image the transformed images obtained by dividing the first target images. (Kim, [0060] After the available object-positioned region is specified in the background image, the object and background combination process is completed by randomly positioning the corresponding object into the specified object-positioned region (S250). The positioning of the object may include randomly positioning the object in the corresponding region regardless of a location and number of objects. The device performs labeling while randomly positioning the corresponding object in the specified object positioned region. In other words, the device stores size information and location information about the object via the labeling, so that when a machine learning program analyzes the image, the program may learn which object is present at which location. In this connection, the category of the object may also be labeled. Codec and other state information may also be labeled. When the object is combined with the background, the object may be inverted, may be rendered in a black and white manner, and may be rotated, or may be subjected to scaling, flipping, perspective transforming, and lighting conditioning. Information on the above processing may be recorded in a label.
Case, [0016] Optionally, determining a placement area of the first background image to place the first cutout object image upon comprises generating a half-tone representation of the first background image.).
The reasoning for combination of Case and Kim is the same as described in Claim 1.
Regarding Claim 7. The combination of Case and Kim further teaches The non-transitory storage medium according to claim 1, wherein the set of the supervised data further comprises positional information about the second target image with respect to the background image (Kim, [0059] … According to the determination result, the device may divide the input background image into one or more regions to specify a region in which the object is to be positioned. The object background matching table defines an image feature of the background region matching with the object using a plurality of parameters related to the image.
[0060] … In other words, the device stores size information and location information about the object via the labeling, so that when a machine learning program analyzes the image, the program may learn which object is present at which location. In this connection, the category of the object may also be labeled. Codec and other state information may also be labeled. When the object is combined with the background, the object may be inverted, may be rendered in a black and white manner, and may be rotated, or may be subjected to scaling, flipping, perspective transforming, and lighting conditioning. Information on the above processing may be recorded in a label.
Case, [0106] Returning to the example of FIG. 3, once the aircraft cutout image has been superimposed onto the background image at step 312, the process can move to step 314 wherein the synthetic training image is automatically annotated with the location of the aircraft in the image for the purpose of providing the information to the supervised training process. In one or more examples, annotating the training image can include indicating a location of a bounding box where the aircraft is located (for instance by appending the information to the metadata of the image) so that during the supervised training process, the machine learning classifier can learn from the image by knowing where the aircraft in the image being used to train the classifier is located. In this way, in addition to multiplying the amount of training data available to train a machine learning classifier, the process 300 can also reduce the amount of effort required to annotate a training data set.).
The reasoning for combination of Case and Kim is the same as described in Claim 1.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Price et al (US20230112186), abstract, the invention teaches an alpha matting system that utilizes a deep learning model to generate alpha mattes for digital images utilizing an alpha-range classifier function. More specifically, in various implementations, the alpha matting system builds and utilizes an object mask neural network having a decoder that includes an alpha-range classifier to determine classification probabilities for pixels of a digital image with respect to multiple alpha-range classifications. In addition, the alpha matting system can utilize a refinement model to generate the alpha matte from the pixel classification probabilities with respect to the multiple alpha-range classifications.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XIN SHENG whose telephone number is (571)272-5734. The examiner can normally be reached M-F 9:30AM-3:30PM 6:00PM-8:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jason Chan can be reached at 5712723022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Xin Sheng/ Primary Examiner, Art Unit 2619
Prosecution Insights