CTNF 18/571,687 CTNF 100747 DETAILED ACTION This office action is responsive to applicant’s communication filed 04/13/2026. Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Response to Arguments Applicant’s arguments with respect to claim(s) 1, 6, and 11 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim(s ) 1-7 and 16-18 is /are rejected under 35 U.S.C. 103 as being unpatentable over Ro well et al. (US 20200342652 A1; hereinafter "Rowell") in view of Sharma et al. (US 20210201474 A1; hereinafter "Sharma"). Re garding claim 1, Rowell teaches: A computing device, comprising: a processor; and a memory ([0039] “The image computing device 100 may further include, a communication unit 110, memory 112, a processor 113, a storage device 111, and a display 116.”) communicatively coupled to the processor and storing executable instructions ([0040] “The memory 112 may store the code, routines, and data necessary for the imaging system 125 to provide its functionality. The memory 112 is coupled to the interconnect 115 for communication with the other components.”) that when executed cause the processor to: generate multiple synthetic images of an object ([0058] describes object selection: “After generating the background image and adding the background image to a background portion of the image scene, the image scene generator 124 populates the image scene by adding foreground objects to the foreground portion of the image scene.”; fig. 3 shows and [0067] describes the generation of multiple images of the same scene from various angles: “Depending on the position of the virtual camera module 204 within the 3D scene 200, different points included in the 3D object will be projected onto a 2D surface. Therefore, the image projector 126 may generate multiple camera views for each image scene with each camera view including a unique perspective of the image scene. In one embodiment, the image projector 126 generates a plurality of camera views by modifying one or more camera coordinates 210 included in a camera setting file. Depending on the application of the synthetic images produced by the synthetic image generation module 102, the camera coordinates 210 may be consistently and/or incrementally varied for each image scene generated during production of a particular synthetic image dataset to ensure the synthetic image dataset includes synthetic images capturing a consistent set of camera views for every scene. Alternatively, the camera coordinates 210 may be modified randomly by the image projector 126 to introduce additional variation into a synthetic image dataset.”; fig. 4 shows the generation of multiple images of the same scene with various image augmentations) , wherein the object is based on defined object parameters ([0060] “The 3D models and other foreground objects selected by the 3D model selection routine may be specific to the intended CV application of the synthetic images produced by the synthetic image generation system 101. A 3D model arrangement routine executed by the image scene generator 124 to position the 3D models and other foreground objects in a foreground portion of the scene may also be specific to the application of the synthetic images. The 3D model arrangement routine may define the horizontal and vertical location and depth of each object (e.g., 3D models and other foreground objects) in a scene. In one embodiment, to produce image scenes for generating synthetic images to train machine learning systems for object detection in autonomous vehicles, the image scene generator 124 selects objects commonly found near and/or on roads (e.g., cars, trucks, buses, trees, buildings, humans, animals, traffic lights, traffic signs, etc.) and positions the objects to generate a scene resembling the point of view from a car driving on the road. One or more affine transformations may also be applied to one or more background and/or foreground objects in the scene to simulate motion.”; also see paragraphs [0111] to [0113] describing “scene metadata”) and wherein respective ones of the multiple synthetic images are based on randomized visual parameters applied to the object (fig. 4; [0091] “In some embodiments, it may be desirable to modify synthetic images provided by the synthetic image generation module 102 with noise and other defects. To increase the amount of variation in a synthetic image dataset, the image augmentation engine 104 may add noise and distortion to synthetic images.”; [0099] teaches image augmentation based on randomness: “In one embodiment, the image augmentation engine 104 specifies a range of augmentation values for each augmentation operation then determines the augmentation to apply to each raw synthetic image 401 by sampling values within the range. For example, the image augmentation engine 104 may provide a range for a transformation augmentation operation between −20% and 20% of the image width for x and y, a range for a rotation augmentation operation from −17° to 17°, a range for a scaling rotation augmentation operation between 0.9 and 2.0, a range for a contrast augmentation operation from −0.8 to 0.4, and a range for multiplicative color changes to each RGB channel from 0.5 to 2. Values may be randomly , incrementally, or otherwise sampled from these ranges.”) ; generate, based on the defined object parameters and the randomized visual parameters, annotations of the object in the multiple synthetic images ([0027] “The synthetic image generation system also includes a synthetic image indexing module that provides image metadata for each synthetic image. The image metadata comprises synthetic image characteristics including scene composition (i.e., background, foreground objects, object arrangement, textures , etc.), camera settings (e.g., camera intrinsics, camera extrinsics, camera calibration metadata, etc.), camera capture settings (e.g., zoom, focus, baseline, zero disparity plane depth, lighting conditions, etc.), and image augmentations. By referencing image metadata during the dataset generation, the synthetic image generation system avoids producing duplicate images and provides precise control of synthetic image characteristics.”; also see paragraphs [0029] and [0125]) ; and using the multiple synthetic images and the annotations, train a machine-learning (ML) model to detect the object using the multiple synthetic images and annotations ([0060] “In one embodiment, to produce image scenes for generating synthetic images to train machine learning systems for object detection in autonomous vehicles, the image scene generator 124 selects objects commonly found near and/or on roads (e.g., cars, trucks, buses, trees, buildings, humans, animals, traffic lights, traffic signs, etc.) and positions the objects to generate a scene resembling the point of view from a car driving on the road.”; [0121] describes how the training data includes annotations: “Training datasets including collections of synthetic image data generated by the synthetic image generation system 100 are stored in a training database 905. The training database 905 may also include synthetic image data metadata describing characteristics of synthetic images and additional image data channels as well as scene metadata describing attributes of image scenes captured in synthetic images and additional image data channels.”) . Rowell does not explicitly teach: wherein the object is based on defined object parameters indicating relative positions of individual parts of the object simulating a defect of the object , or train a machine-learning (ML) model to detect the object and the individual parts of the object for identifying the defect . Sharma teaches: generate multiple synthetic images of an object, wherein the object is based on defined object parameters indicating relative positions of individual parts of the object simulating a defect of the object ([0025] teaches generating synthetic images of various configurations of component assemblies, including correct and incorrect (defective) variations: “The various example embodiments described herein provide a convenient way to solve this problem by generating synthetic machine learning training images using a 3D engine as part of the synthetic training data generation system. Inside this engine, computer-aided design (CAD) models for the different sub-components of the component assembly can be virtually assembled and rendered into the various component assembly variants. Then, these various component assembly variants can be rendered under a variety of different lighting conditions, various camera settings or angles, different virtual backgrounds, and the like. In this manner, virtually-generated component assembly variants can be rendered under a variety of conditions and poses. Any number of images of the component assembly variants can be generated. These images of the component assembly variants representing synthetic machine learning training images can be used to train a machine learning system to recognize compliant and non-compliant component assemblies.”; figs. 1-6 show examples; [0032] teaches generating synthetic images with defects added to individual components: “Similarly, images of various types of component defects and their variations can be obtained from selected images of previously manufactured components. Once the visual structure of these defects is abstracted from these selected images, the visual structure of these component defects can be virtually simulated or extracted and added into images of the compliant components. In this manner, virtual defects can be added to images of compliant components to produce synthetically or virtually-generated images of non-compliant components. One advantage of this approach is that the visual structure of component defects can be obtained from a small number of images of defective physical components. These sample images of component defects can be used to produce a variety of different synthetically or virtually-generated images of defects, wherein the defects can be varied in size, orientation, location, quantity, and the like. This variety of different synthetically or virtually-generated images of defects can be added to images of the compliant components to produce a variety of different images of defective or non-compliant components. A large variety and quantity of these different images of defective or non-compliant components can be synthetically generated in this manner. This large set of different synthetically generated images of defective or non-compliant components can be used as a training dataset to train a machine learning system to detect compliant and non-compliant manufactured components.”; fig. 11 shows examples), and train a machine-learning (ML) model to detect the object and the individual parts of the object for identifying the defect ([0027] “These synthetically or virtually-generated training images can then be used to train a machine learning system, which can then detect each sub-component of the component assembly based on the variations presented by the synthetic machine learning training images. The machine learning system can also classify each detected sub-component into its particular variant based on the variations presented by the synthetic machine learning training images. In this manner, the machine learning system can be trained by the synthetically-generated training images to detect the presence or absence of one or more sub-components of a component assembly. The proper configuration of the component assembly can verified by checking the results of the machine learning system against the expected or desired results for a particular component assembly.”); [0032] “This large set of different synthetically generated images of defective or non-compliant components can be used as a training dataset to train a machine learning system to detect compliant and non-compliant manufactured components.”). Rowell and Sharma are both analogous to the claimed invention because they are in the same field of synthetic image generation for training a machine learning model. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Rowell with the teachings of Sharma to incorporate the ability to generate images of various configurations of components of an object, including defective components, including generating annotations for the individual components. The motivation would have been to apply the invention toward inspection and quality control in manufacturing, as discussed in paragraph [0024]. Regarding claim 2, the combination of Rowell in view of Sharma teaches: The computing device of claim 1, wherein the object comprises a geometric shape (Rowell [0058] “In some embodiments, foreground objects are selected from a 3D model library including 36,000 or more 3D objects including chairs, tables, cars, animals, faces, etc. In other embodiments, foreground objects include light sources illuminating image scenes to provide light captured by virtual camera modules during generation of synthetic images. 3D model foreground objects may comprise a three dimensional mesh structure comprising points and/or a polygon mesh outlining a specific shape .”) . Regarding claim 3, the combination of Rowell in view of Sharma teaches: The computing device of claim 1, wherein the randomized visual parameters comprise at least one of lighting, object pose, object orientation, or background image (Rowell [0057] “Background images are added to a background portion of the image scene by the image scene generator 124…The image scene generator 124 may assemble background images by forming a large textured ground plane comprising a solid color, gradient color, or textured 2D image overlaid with a selection of static background objects having shapes randomly chosen from cubes, polyhedrons, cuboids, and cylinders. To add variation to the background image, the synthetic image generation module 102 may scale, rotate, texture, and/or deform one or more objects before placing the object(s) in the 2D image.”) . Regarding claim 4, the combination of Rowell in view of Sharma teaches: The computing device of claim 1, wherein the multiple synthetic images comprise photorealistic images (Rowell [0054] “The parameters included in camera settings files may be setup identical to an actual camera device in order to create camera views capturing a realistic, camera specific representation of an image scene perspective. By including parameters for camera intrinsics, camera calibration metadata, and camera capture settings in the camera settings file that are identical to actual camera device, the synthetic image generation module 102 may generate synthetic images simulating capture performance of the actual camera device under different capture conditions.”; [0063] “In other embodiments, the image scene generator 124 selects a texture file associated with an object in the scene to give the object a realistic appearance.”) generated by a 3D rendering engine (Rowell [0064] “The image projector 126 further processes image scenes from the image scene generator 124 to generate camera views rendered as synthetic images by the graphics rendering engine 128. In some embodiments, the synthetic image generation module 102 produces many synthetic images for each created scene. Each synthetic image captures the scene from a unique perspective by generating a camera view (i.e., a 2D projection) of a 3D scene.”) . Regarding claim 5, the combination of Rowell in view of Sharma teaches: The computing device of claim 1, wherein the defined object parameters define an object type (Rowell [0111] “Scene metadata files may also define the number of objects to place in a scene and/or list specific objects files or object types to incorporate in a scene.”) . Regarding claim 16, the combination of Rowell in view of Sharma teaches: The computing device of claim 1, wherein the defined object parameters include dimensions of the object ([0022] “The systems and methods of the invention also enable more precise scene construction wherein exact values for scene dimensions, object sizes , object depths, and other scene characteristics are known and customizable throughout scene construction.”) . Regarding claim 17, the combination of Rowell in view of Sharma teaches: The computing device of claim 1, wherein the ML model includes at least one of a convolutional neural network (CNN), a recurrent neural network (RNN), or a variant thereof (Rowell [0134] “Machine learning systems communicatively coupled to—and/or included in—the machine learning service 105 include rules based classification algorithms, neural networks, and deep learning methods. More specifically, Naïve Bayes classification algorithms, decision tree classification algorithms, convolutional neural networks (CNNs), convolutional recurrent neural networks (CRNNs), hierarchical recurrent convolutional neural networks (HRNN), and HRNNs with attention vectors implemented in a machine learning framework (e.g., Keras, Scikitlearn, MXNet, or Tensorflow).”) . Regarding claim 6, Rowell teaches: A non-transitory computer-readable storage medium comprising instructions executable by a processor (fig. 1 memory 112; [0040] “A memory 112 may include a non-transitory memory that stores data for providing the functionality described herein… The memory 112 may store the code, routines, and data necessary for the imaging system 125 to provide its functionality.”) to: generate multiple synthetic images of an object ([0058] describes object selection: “After generating the background image and adding the background image to a background portion of the image scene, the image scene generator 124 populates the image scene by adding foreground objects to the foreground portion of the image scene.”; fig. 3 shows and [0067] describes the generation of multiple images of the same scene from various angles: “Depending on the position of the virtual camera module 204 within the 3D scene 200, different points included in the 3D object will be projected onto a 2D surface. Therefore, the image projector 126 may generate multiple camera views for each image scene with each camera view including a unique perspective of the image scene. In one embodiment, the image projector 126 generates a plurality of camera views by modifying one or more camera coordinates 210 included in a camera setting file. Depending on the application of the synthetic images produced by the synthetic image generation module 102, the camera coordinates 210 may be consistently and/or incrementally varied for each image scene generated during production of a particular synthetic image dataset to ensure the synthetic image dataset includes synthetic images capturing a consistent set of camera views for every scene. Alternatively, the camera coordinates 210 may be modified randomly by the image projector 126 to introduce additional variation into a synthetic image dataset.”; fig. 4 shows the generation of multiple images of the same scene with various image augmentations) including: generating the multiple synthetic images by applying randomized visual parameters to the object (fig. 4; [0091] “In some embodiments, it may be desirable to modify synthetic images provided by the synthetic image generation module 102 with noise and other defects. To increase the amount of variation in a synthetic image dataset, the image augmentation engine 104 may add noise and distortion to synthetic images.”; [0099] teaches image augmentation based on randomness: “In one embodiment, the image augmentation engine 104 specifies a range of augmentation values for each augmentation operation then determines the augmentation to apply to each raw synthetic image 401 by sampling values within the range. For example, the image augmentation engine 104 may provide a range for a transformation augmentation operation between −20% and 20% of the image width for x and y, a range for a rotation augmentation operation from −17° to 17°, a range for a scaling rotation augmentation operation between 0.9 and 2.0, a range for a contrast augmentation operation from −0.8 to 0.4, and a range for multiplicative color changes to each RGB channel from 0.5 to 2. Values may be randomly , incrementally, or otherwise sampled from these ranges.”) ; generate, based on the randomized visual parameters, annotations for the multiple synthetic images ([0027] “The synthetic image generation system also includes a synthetic image indexing module that provides image metadata for each synthetic image. The image metadata comprises synthetic image characteristics including scene composition (i.e., background, foreground objects, object arrangement, textures , etc.), camera settings (e.g., camera intrinsics, camera extrinsics, camera calibration metadata, etc.), camera capture settings (e.g., zoom, focus, baseline, zero disparity plane depth, lighting conditions, etc.), and image augmentations. By referencing image metadata during the dataset generation, the synthetic image generation system avoids producing duplicate images and provides precise control of synthetic image characteristics.”; also see paragraphs [0029] and [0125]) ; and using the multiple synthetic images and the annotations, train a machine-learning (ML) model to detect the object ([0060] “In one embodiment, to produce image scenes for generating synthetic images to train machine learning systems for object detection in autonomous vehicles, the image scene generator 124 selects objects commonly found near and/or on roads (e.g., cars, trucks, buses, trees, buildings, humans, animals, traffic lights, traffic signs, etc.) and positions the objects to generate a scene resembling the point of view from a car driving on the road.”; [0121] describes how the training data includes annotations: “Training datasets including collections of synthetic image data generated by the synthetic image generation system 100 are stored in a training database 905. The training database 905 may also include synthetic image data metadata describing characteristics of synthetic images and additional image data channels as well as scene metadata describing attributes of image scenes captured in synthetic images and additional image data channels.”) . Rowell does not explicitly teach: generating multiple synthetic images of multiple types of defects of an object; simulating the multiple types of defects based on relative positions of individual parts of the object; generating annotations based on the individual parts of the object ; or training a machine learning model to detect the individual parts of the object for identifying the multiple types of defects . Sharma teaches: generating multiple synthetic images of multiple types of defects of an object ([0032] teaches generating synthetic images with defects added to individual components: “Similarly, images of various types of component defects and their variations can be obtained from selected images of previously manufactured components. Once the visual structure of these defects is abstracted from these selected images, the visual structure of these component defects can be virtually simulated or extracted and added into images of the compliant components. In this manner, virtual defects can be added to images of compliant components to produce synthetically or virtually-generated images of non-compliant components. One advantage of this approach is that the visual structure of component defects can be obtained from a small number of images of defective physical components. These sample images of component defects can be used to produce a variety of different synthetically or virtually-generated images of defects, wherein the defects can be varied in size, orientation, location, quantity, and the like. This variety of different synthetically or virtually-generated images of defects can be added to images of the compliant components to produce a variety of different images of defective or non-compliant components. A large variety and quantity of these different images of defective or non-compliant components can be synthetically generated in this manner. This large set of different synthetically generated images of defective or non-compliant components can be used as a training dataset to train a machine learning system to detect compliant and non-compliant manufactured components.”; fig. 11 shows examples of multiple types of defects); simulating the multiple types of defects based on relative positions of individual parts of the object ([0025] teaches generating synthetic images of various configurations of component assemblies, including correct and incorrect (defective) variations: “The various example embodiments described herein provide a convenient way to solve this problem by generating synthetic machine learning training images using a 3D engine as part of the synthetic training data generation system. Inside this engine, computer-aided design (CAD) models for the different sub-components of the component assembly can be virtually assembled and rendered into the various component assembly variants. Then, these various component assembly variants can be rendered under a variety of different lighting conditions, various camera settings or angles, different virtual backgrounds, and the like. In this manner, virtually-generated component assembly variants can be rendered under a variety of conditions and poses. Any number of images of the component assembly variants can be generated. These images of the component assembly variants representing synthetic machine learning training images can be used to train a machine learning system to recognize compliant and non-compliant component assemblies.”; figs. 1-6 show examples); and training a machine learning model to detect the individual parts of the object for identifying the multiple types of defects ([0027] “These synthetically or virtually-generated training images can then be used to train a machine learning system, which can then detect each sub-component of the component assembly based on the variations presented by the synthetic machine learning training images. The machine learning system can also classify each detected sub-component into its particular variant based on the variations presented by the synthetic machine learning training images. In this manner, the machine learning system can be trained by the synthetically-generated training images to detect the presence or absence of one or more sub-components of a component assembly. The proper configuration of the component assembly can verified by checking the results of the machine learning system against the expected or desired results for a particular component assembly.”); [0032] “This large set of different synthetically generated images of defective or non-compliant components can be used as a training dataset to train a machine learning system to detect compliant and non-compliant manufactured components.”). Rowell and Sharma are both analogous to the claimed invention because they are in the same field of synthetic image generation for training a machine learning model. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Rowell with the teachings of Sharma to incorporate the ability to generate images of various configurations of components of an object, including defective components. In particular, the invention of Rowell teaches the inclusion of metadata for all details concerning the selection, positioning, and orientation of objects; therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Rowell with the teachings of Sharma to also include metadata describing the individual subcomponents of an object for each generated image. The motivation would have been to apply the invention toward inspection and quality control in manufacturing, as discussed in paragraph [0024]. Regarding claim 7, the combination of Rowell in view of Sharma teaches: The non-transitory computer-readable storage medium of claim 6, wherein the instructions to generate the multiple synthetic images comprise instructions executable by the processor to: add randomized out-of-focus effects of a camera lens to the synthetic images (Rowell [0158] “In one embodiment, camera blur 750 and lens flare 752 effects may be included in augmentation settings 550 for virtual cameras generating synthetic images under specific capture conditions. One camera blur effect 750 includes motion blur (i.e., a streaking effect of some captured objects) resulting from long exposure and/or rapid movement of objects in a scene portion captured in a synthetic image. In other embodiments, a Camera blur effect 750 may be a Gaussian blur simulating capture with poor focus and/or a post capture smoothing and/or noise reducing function included in an actual camera device.”; [0099] describes how parameters for image augmentation are generated by randomly sampling from given ranges) ; and simulate motion blurring in the synthetic images, wherein each of the synthetic images includes different randomized motion blurring (Rowell [0091] “In some embodiments, it may be desirable to modify synthetic images provided by the synthetic image generation module 102 with noise and other defects. To increase the amount of variation in a synthetic image dataset, the image augmentation engine 104 may add noise and distortion to synthetic images. By adding other effects (e.g., lens flare, lens distortion, motion blur , and Gaussian blur), the image augmentation engine 104 may also simulate realistic capture errors that commonly occur under certain conditions with specific camera hardware.”, [0099] describes how parameters for image augmentation are generated by randomly sampling from given ranges) . Regarding claim 18, the combination of Rowell in view of Sharma teaches: The non-transitory computer-readable storage medium of claim 6, wherein the instructions are further executable by the processor to select a background image, wherein the multiple synthetic images include the background image (Rowell [0057] “Background images are added to a background portion of the image scene by the image scene generator 124.”; the remainder of the paragraph explains the process) . 07-22-aia AIA Claim (s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rowell (US 20200342652 A1) in view of Sharma (US 20210201474 A1) as applied to claim 6 above, and further in view of Jacobs et al. (US 20230351560 A1, hereinafter "Jacobs") . Regarding claim 8, the combination of Rowell in view of Sharma teaches: The non-transitory computer-readable storage medium of claim 6, but does not explicitly teach: wherein the instructions to generate the multiple synthetic images comprise instructions executable by the processor to: simulate shadows cast on the object in the synthetic images, wherein each of the synthetic images includes randomized shadows cast on the object. Jacobs teaches: wherein the instructions to generate the multiple synthetic images comprise instructions executable by the processor to: simulate shadows cast on the object in the synthetic images ([0027] “The first well-lit image and the second shadowed image are then processed via a two-dimensional image-based data generation pipeline to create a synthetic shadowed image. The synthetic shadowed image could be used to train the machine learning model. A series of steps are taken within the data pipeline. Once a well-lit image and a shadowed image of the subject are produced, the images can be combined in the pipeline using a mask. In an example embodiment, the mask may control the transparency/opacity of an image with respect to a background image. In some examples, a plurality of masks could correspond to different shadow shapes and/or occluder shapes. In various embodiments, the shadowed image could be layered over the well-lit image and the mask can be applied to the shadowed image. Such an arrangement could form a synthetic shadow by retaining a shadowed image portion that corresponds to a shape of the mask. In such scenarios, the remainder of the shadowed image may appear transparent so that the well-lit image is observable. In such a manner, this process forms a synthetically shadowed image.”) , wherein each of the synthetic images includes randomized shadows cast on the object ([0030] “In some embodiments, the shape of the mask may be the principal identifier that the machine learning model could be trained to recognize among a plurality of different shaped shadows. The shape of the mask could be used to approximate real world shapes. A shadow mask may be produced by using 3D models of occluders and facial geometry to project shadows based on features such as the contours of the face. A shadow mask may also be produced from hand drawn 2D masks, randomly synthetically generated 2D masks , or a combination of any of the above with localized color and sharpness variations to emulate real world phenomena such as subsurface scattering in skin, spatially varying lighting environments, or shadow foreshortening.”, [0063] “In some embodiments, the shadow shapes could additionally or alternatively be gathered from two-dimensional shapes 808. For example, the two-dimensional shapes 808 could be hand drawn shapes of shadows, they could be random shapes , they could be traced shapes, they could include outlines from photographs, or they could be produced from any other manner not mentioned in order to obtain a two dimensional shape of a shadow.”) . Jacobs is analogous to the claimed invention because it is in the same field of generating synthetic images to train a machine learning model (Jacobs [0025]-[0027]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the invention of Rowell in view of Sharma with the teachings of Jacobs to add simulated shadows as another option for parametrized, randomizable image augmentation, applying them to the multiple objects found in the images taught by Rowell in view of Sharma. The motivation would have been to improve the training dataset by depicting a wider variety of image conditions . 07-22-aia AIA Claim (s) 9, 10, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rowell (US 20200342652 A1) in view of Sharma (US 20210201474 A1) as applied to claim 6 above, and further in view of Wrenninge et al. (US 20190156151 A1, hereinafter "Wrenninge") . Regarding claim 9, the combination of Rowell in view of Sharma teaches: The non-transitory computer-readable storage medium of claim 6, wherein the instructions to generate the multiple synthetic images comprise instructions executable by the processor to: add noise to the synthetic images (Rowell [0091] “In some embodiments, it may be desirable to modify synthetic images provided by the synthetic image generation module 102 with noise and other defects. To increase the amount of variation in a synthetic image dataset, the image augmentation engine 104 may add noise and distortion to synthetic images.”) . The combination of Rowell in view of Sharma does not explicitly teach: wherein each of the synthetic images includes a randomized type of noise. Wrenninge teaches: wherein each of the synthetic images includes a randomized type of noise ([0069] “Examples of parameters on which augmentation can be based (e.g., utilized to augment a synthetic image) include… simulated sensor or camera parameters (e.g., governing exposure level, sensor dynamic range, sensor black level, light response curve, static noise, temporal noise, shot noise, photon noise, color filter array/CFA arrangement, CFA filter characteristics, demosaicing, etc.), and any other suitable parameters related to varying or altering the synthetic image subsequent to rendering.”, [0041]-[0042] teaches that the value of each parameter is determined by sampling a probability density function associated with the parameter, [0045] teaches that the sampling of the probability density function may be randomized”) . Wrenninge is analogous to the claimed invention because it is in the same field of synthetic image generation for training a machine learning model. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the invention of Rowell in view of Sharma with the teachings of Wrenninge to randomly select a type of noise to be applied to a synthetic image. The motivation would have been to diversify the generated training dataset by increasing the variety of parameters and depicting a wider variety of image conditions. Regarding claim 10, the combination of Rowell in view of Sharma teaches: The non-transitory computer-readable storage medium of claim 6, wherein the instructions to generate the annotations comprise instructions executable by the processor to: determine an object type for the object in the multiple synthetic images (Rowell [0125] “In some embodiments, the image indexing module 108 may index images on metadata fields encoded into the synthetic image at capture by the synthetic image generation module 102. The metadata fields include… image metadata (e.g., type and/or number objects included in an image , depth of background image, depth of a foreground object, maximum depth of a foreground object, number and type image augmentations performed on a parent synthetic image, additional image data rendered for a camera view captured in a parent synthetic image, etc.)”) ; determine part types for the individual parts of the object in the multiple synthetic images (Sharma [0029] “As described above, the synthetic training data generation system 100 of an example embodiment can be configured to virtually assemble models for different sub-components of the component assembly and render the sub-component models into images of various component assembly variants. Each component assembly variant can represent a different sub-component configuration and/or a different view or pose of the component assembly. Images of these variants of the component assemblies with sub-component configurations can be collected into a training dataset and used to train a machine learning system.”) ; The invention of Rowell teaches the inclusion of metadata for all details concerning the selection, positioning, and orientation of objects; therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Rowell in view of Sharma with the additional teachings of Sharma to also include metadata describing the individual subcomponents of an object for each generated image. The motivation would have been to improve neural network training by providing detailed ground truth information for supervised learning. The combination of Rowell in view of Sharma does not explicitly teach: wherein the instructions to generate the annotations comprise instructions executable by the processor to: determine a bounding box for the object in each of the multiple synthetic images; and generate a segmentation mask for the object in each of the multiple synthetic images. Wrenninge teaches: wherein the instructions to generate the annotations comprise instructions executable by the processor to: determine a bounding box for the object in each of the multiple synthetic images (Wrenninge [0066] “The output of Block S300 preferably includes a two dimensional synthetic image that realistically depicts a realistic 3D scene. The synthetic image defines a set of pixels, and each pixel is preferably labeled with the object depicted by the pixel (e.g., intrinsically labeled based on the parameters used to generate the object rendered in the image). In this manner, a “pixel-perfect” intrinsically annotated synthetic image can be created. In alternative variations, labelling can be performed on a basis other than a pixel-by-pixel basis; for example, labelling can include automatically generating a bounding box around objects depicted in the image , a bounding polygon of any other suitable shape, a centroid point, a silhouette or outline, a floating label, and any other suitable annotation, wherein the annotation includes label metadata such as the object class and/or other object metadata.”) ; and generate a segmentation mask for the object in each of the multiple synthetic images (Wrenninge fig. 5 shows generated segmentation mask, [0066] “Semantic segmentation can be performed with any suitable level of granularity, and labeled or annotated pixels can be grouped by any category or subcategory of label or annotation (e.g., defined by the parameters used to procedurally define the scene in the image) in producing a segmentation of the image. For example, pixels can be semantically segmented by object class, object subclass, orientation, any other suitable geometric or other parameter as determined in accordance with one or more variations of Block S100, and/or any suitable combination of the aforementioned.”; Rowell [0082] also teaches a segmentation mask, but teaches layers separated by depth rather than separating each individual object) . Wrenninge is analogous to the claimed invention because it is in the same field of synthetic image generation for training a machine learning model. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have modified the invention of Rowell in view of Sharma with the teachings of Wrenninge to generate bounding boxes and segmentation masks for each object in the synthetic images. The motivation would have been to improve neural network training by providing detailed ground truth information for supervised learning. Regarding claim 19, the combination of Rowell in view of Sharma and further in view of Wrenninge teaches: The non-transitory computer-readable storage medium of claim 9, wherein the type of noise includes at least one of a Gaussian noise, a salt-and-pepper noise, a Poisson noise, or a speckle noise (Rowell [0096] “Other image degradations provided by the image augmentation engine 104 include Gaussian noise and Gaussian blur. In some embodiments, the image augmentation engine 104 creates noise augmentation images 406 and Gaussian blur images 408 by applying a Gaussian augmentation operation to a raw synthetic image 401. Gaussian augmentation operations distribute a defined amount of noise (e.g., occlusion or blurring) evenly throughout an entire image.”) . 07-21-aia AIA Claim (s) 11 and 13-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sharma et al. (US 20210201474 A1; hereinafter "Sharma") in view of Rowell et al. (US 20200342652 A1; hereinafter "Rowell") . Regarding claim 11, Sharma teaches: A method, comprising: generating a simulated object from a number of simulated subcomponents, wherein the number of simulated subcomponents are positioned to represent a defect of the simulated object ([0025] teaches generating synthetic images of various configurations of component assemblies, including correct and incorrect (defective) variations: “The various example embodiments described herein provide a convenient way to solve this problem by generating synthetic machine learning training images using a 3D engine as part of the synthetic training data generation system. Inside this engine, computer-aided design (CAD) models for the different sub-components of the component assembly can be virtually assembled and rendered into the various component assembly variants. Then, these various component assembly variants can be rendered under a variety of different lighting conditions, various camera settings or angles, different virtual backgrounds, and the like. In this manner, virtually-generated component assembly variants can be rendered under a variety of conditions and poses. Any number of images of the component assembly variants can be generated. These images of the component assembly variants representing synthetic machine learning training images can be used to train a machine learning system to recognize compliant and non-compliant component assemblies.”; figs. 1-6 show examples; [0032] teaches generating synthetic images with defects added to individual components: “Similarly, images of various types of component defects and their variations can be obtained from selected images of previously manufactured components. Once the visual structure of these defects is abstracted from these selected images, the visual structure of these component defects can be virtually simulated or extracted and added into images of the compliant components. In this manner, virtual defects can be added to images of compliant components to produce synthetically or virtually-generated images of non-compliant components. One advantage of this approach is that the visual structure of component defects can be obtained from a small number of images of defective physical components. These sample images of component defects can be used to produce a variety of different synthetically or virtually-generated images of defects, wherein the defects can be varied in size, orientation, location, quantity, and the like. This variety of different synthetically or virtually-generated images of defects can be added to images of the compliant components to produce a variety of different images of defective or non-compliant components. A large variety and quantity of these different images of defective or non-compliant components can be synthetically generated in this manner. This large set of different synthetically generated images of defective or non-compliant components can be used as a training dataset to train a machine learning system to detect compliant and non-compliant manufactured components.”; fig. 11 shows examples) ; generating multiple synthetic images of the simulated object ([0025] “Inside this engine, computer-aided design (CAD) models for the different sub-components of the component assembly can be virtually assembled and rendered into the various component assembly variants. Then, these various component assembly variants can be rendered under a variety of different lighting conditions, various camera settings or angles, different virtual backgrounds, and the like.”) ; and based on the multiple synthetic images and the annotations, training a machine-learning (ML) model to detect an observed object and subcomponents of the observed object in images captured by a camera for identifying the defect ([0027] “These synthetically or virtually-generated training images can then be used to train a machine learning system, which can then detect each sub-component of the component assembly based on the variations presented by the synthetic machine learning training images. The machine learning system can also classify each detected sub-component into its particular variant based on the variations presented by the synthetic machine learning training images. In this manner, the machine learning system can be trained by the synthetically-generated training images to detect the presence or absence of one or more sub-components of a component assembly.”) . Sharma does not explicitly teach: generating multiple synthetic images of the simulated object based on randomized visual parameters ; or generating, based on information from the number of simulated subcomponents and the randomized visual parameters, annotations for the multiple synthetic images . Rowell teaches: generating multiple synthetic images of the simulated object based on randomized visual parameters (fig. 4; [0091] “In some embodiments, it may be desirable to modify synthetic images provided by the synthetic image generation module 102 with noise and other defects. To increase the amount of variation in a synthetic image dataset, the image augmentation engine 104 may add noise and distortion to synthetic images.”; [0099] teaches image augmentation based on randomness: “In one embodiment, the image augmentation engine 104 specifies a range of augmentation values for each augmentation operation then determines the augmentation to apply to each raw synthetic image 401 by sampling values within the range. For example, the image augmentation engine 104 may provide a range for a transformation augmentation operation between −20% and 20% of the image width for x and y, a range for a rotation augmentation operation from −17° to 17°, a range for a scaling rotation augmentation operation between 0.9 and 2.0, a range for a contrast augmentation operation from −0.8 to 0.4, and a range for multiplicative color changes to each RGB channel from 0.5 to 2. Values may be randomly , incrementally, or otherwise sampled from these ranges.”); and generating, based on information from the randomized visual parameters, annotations for the multiple synthetic images ([0027] “The synthetic image generation system also includes a synthetic image indexing module that provides image metadata for each synthetic image. The image metadata comprises synthetic image characteristics including scene composition (i.e., background, foreground objects, object arrangement, textures, etc.), camera settings (e.g., camera intrinsics, camera extrinsics, camera calibration metadata, etc.), camera capture settings (e.g., zoom, focus, baseline, zero disparity plane depth, lighting conditions, etc.), and image augmentations. By referencing image metadata during the dataset generation, the synthetic image generation system avoids producing duplicate images and provides precise control of synthetic image characteristics.”; also see paragraphs [0029] and [0125]). Sharma and Rowell are both analogous to the claimed invention because they are in the same field of synthetic image generation for training a machine learning model. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Sharma with the teachings of Rowell to incorporate the ability to generate synthetic images with randomized parameters and to generate annotations for the synthetic images. In particular, the invention of Rowell teaches the inclusion of metadata for all details concerning the selection, positioning, and orientation of objects; therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the invention of Sharma with the teachings of Rowell to also include metadata describing the individual subcomponents of an object for each generated image. The motivation would have been to increase the variety in training data in a synthetic dataset, and to improve neural network training by providing detailed ground truth information for supervised learning. Regarding claim 13, the combination of Sharma in view of Rowell teaches: The method of claim 11, wherein generating the annotations for the multiple synthetic images comprises identifying a subcomponent of the simulated object based on a part type (Sharma [0027] “These synthetically or virtually-generated training images can then be used to train a machine learning system, which can then detect each sub-component of the component assembly based on the variations presented by the synthetic machine learning training images. The machine learning system can also classify each detected sub-component into its particular variant based on the variations presented by the synthetic machine learning training images. In this manner, the machine learning system can be trained by the synthetically-generated training images to detect the presence or absence of one or more sub-components of a component assembly.”) . It would have been obvious to modify the invention of Sharma in view of Rowell, which teaches generating annotations as discussed for claim 11, in light of the above teachings to generate annotations for individual part types in order to more easily achieve the described results Regarding claim 14, the combination of Sharma in view of Rowell teaches: The method of claim 11, further comprising running the ML model to detect the observed object and the subcomponents of the observed object in an image captured by a camera (Sharma [0038] “This large set of different synthetically generated images of the component can be used as a training dataset to train a machine learning system to detect and count particular individual components. It should also be noted that once a component is detected, its size can also be estimated if the camera hardware and pose are known relative to the component. After the trained machine learning system detects and counts particular individual components, the component count can be compared to a count corresponding to a compliant set of components.”) . Regarding claim 15, the combination of Sharma in view of Rowell teaches: The method of claim 14, further comprising detecting the defect in the observed object based on the detected subcomponents of the observed object in the image (Sharma [0038] “In this manner, the machine learning system trained with synthetically generated component images can be used to detect non-compliant sets of manufactured components.”) . 07-22-aia AIA Claim (s) 12 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Sharma (US 20210201474 A1) in view of Rowell (US 20200342652 A1) as applied to claim 11 above, and further in view of Lundeen et al. (US 20220374834 A1, hereinafter "Lundeen") . Regarding claim 12, the combination of Sharma in view of Rowell teaches: The method of claim 11, but does not explicitly teach: wherein the simulated object comprises a shipping package and the number of simulated subcomponents comprise components of the shipping package. Lundeen teaches: wherein the simulated object comprises a shipping package ([0062] explains the idea of using machine learning to identify shipping packages and generate 3D bounding boxes for the packages. [0063]-[0064] describe one approach to doing this, which involves generating photorealistic simulated images of scenes including 3D models of shipping packages, in which the shipping boxes are annotated with their ground truth 3D bounding box data). Lundeen also teaches that shipping packages may have subcomponents including a label and/or one or more fiducial markers such as AprilTags or a QR code (fig. 2B, [0055]). Lundeen is analogous to the claimed invention because it is in the same field of generating training data to train a neural network in object recognition. Though Lundeen does not explicitly teach that the shipping package subcomponents are depicted in the simulated 3D model, it would have been obvious to one of ordinary skill in the art to modify the invention of Sharma in view of Rowell, in which simulated objects are generated and annotated based on their constituent subcomponents, with the teachings of Lundeen to apply the invention specifically toward shipping packages, including modeling their separate subcomponents. The motivation would have been to be able to use the invention to inspect packages in a similar manner as the inspections of Sharma, including helping to track and sort packages for delivery, as taught by Lundeen (“Background” section, [0001]-[0003]). Regarding claim 20, the combination of Sharma in view of Rowell and further in view of Lundeen teaches: The method of claim 12, wherein the defect includes at least one of a no-pattern- found defect, a side glue defect, a wrong color defect, a dented surface defect, a large gap defect, a label alignment defect, or a side skew defect (Sharma fig. 13 shows a dented surface defect) . Conclusion Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN STATZ whose telephone number is (571)272-6654. The examiner can normally be reached Mon-Fri 8am-5pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571)272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /BENJAMIN TOM STATZ/ Examiner, Art Unit 2611 /TAMMY PAIGE GODDARD/ Supervisory Patent Examiner, Art Unit 2611 Application/Control Number: 18/571,687 Page 2 Art Unit: 2611 Application/Control Number: 18/571,687 Page 3 Art Unit: 2611 Application/Control Number: 18/571,687 Page 4 Art Unit: 2611 Application/Control Number: 18/571,687 Page 5 Art Unit: 2611 Application/Control Number: 18/571,687 Page 6 Art Unit: 2611 Application/Control Number: 18/571,687 Page 7 Art Unit: 2611 Application/Control Number: 18/571,687 Page 8 Art Unit: 2611 Application/Control Number: 18/571,687 Page 9 Art Unit: 2611 Application/Control Number: 18/571,687 Page 10 Art Unit: 2611 Application/Control Number: 18/571,687 Page 11 Art Unit: 2611 Application/Control Number: 18/571,687 Page 12 Art Unit: 2611 Application/Control Number: 18/571,687 Page 13 Art Unit: 2611 Application/Control Number: 18/571,687 Page 14 Art Unit: 2611 Application/Control Number: 18/571,687 Page 15 Art Unit: 2611 Application/Control Number: 18/571,687 Page 16 Art Unit: 2611 Application/Control Number: 18/571,687 Page 17 Art Unit: 2611 Application/Control Number: 18/571,687 Page 18 Art Unit: 2611 Application/Control Number: 18/571,687 Page 19 Art Unit: 2611 Application/Control Number: 18/571,687 Page 20 Art Unit: 2611 Application/Control Number: 18/571,687 Page 21 Art Unit: 2611 Application/Control Number: 18/571,687 Page 22 Art Unit: 2611 Application/Control Number: 18/571,687 Page 23 Art Unit: 2611 Application/Control Number: 18/571,687 Page 24 Art Unit: 2611 Application/Control Number: 18/571,687 Page 25 Art Unit: 2611 Application/Control Number: 18/571,687 Page 26 Art Unit: 2611 Application/Control Number: 18/571,687 Page 27 Art Unit: 2611 Application/Control Number: 18/571,687 Page 28 Art Unit: 2611 Application/Control Number: 18/571,687 Page 29 Art Unit: 2611 Application/Control Number: 18/571,687 Page 30 Art Unit: 2611 Application/Control Number: 18/571,687 Page 31 Art Unit: 2611 Application/Control Number: 18/571,687 Page 32 Art Unit: 2611 Application/Control Number: 18/571,687 Page 33 Art Unit: 2611 Application/Control Number: 18/571,687 Page 34 Art Unit: 2611
Prosecution Insights