CTNF 18/781,937 CTNF 87262 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claims 1-20 are pending in the application. Claim Objections Claim 10 “the one or more physical objects” and “the mapped one or more features” has no antecedent basis. For prior art rejection, Examiner considers “the one or more physical objects” “the one or more features”. Claim 20 has similar issue. Double Patenting 08-33 AIA The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg , 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman , 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi , 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum , 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel , 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington , 418 F.2d 528, 163 USPQ 644 (CCPA 1969). A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b). The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13. The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA/25, or PTO/AIA/26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer. 08-34 AIA Claim s 1 and 11 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim s 1 and 12 respectively of U.S. Patent No. 12,045,955 B2 . Although the claims at issue are not identical, they are not patentably distinct from each other because the following reasons . Listed in the following table is a limitation-to-limitation comparison of the examined claim and the conflicting claim. Application being examined 18/781,937 (hereafter ‘937 application) Conflicting Patent 12,045,955 B2 (hereafter ‘955 patent) Claim 1 1. A method for creating panoramic imagery, the method comprising: obtaining, by one or more processors, first panoramic imagery depicting a geographic area; obtaining, by the one or more processors, an image including a depiction of an environmental condition or a transient condition absent from the first panoramic imagery; and transforming, by the one or more processors and using a machine learning model, the first panoramic imagery into second panoramic imagery depicting the environmental condition or the transient condition overlaying the first panoramic imagery. Claim 1 1. A method for providing panoramic imagery, the method comprising: obtaining, by one or more processors, first panoramic imagery depicting a geographic area at a first time; obtaining, by the one or more processors, an image depicting at least a portion of the geographic area at a second time later than the first time, the image further depicting one or more features not depicted in the first panoramic imagery; and transforming, by the one or more processors and using a machine learning model, the first panoramic imagery into second panoramic imagery depicting the one or more features and including at least a portion of the first panoramic imagery. Claim 11 11. A system for providing panoramic imagery, the system comprising: one or more processors; and a non-transitory memory storing instructions that, when executed by the one or more processors, cause the system to: obtain first panoramic imagery depicting a geographic area; obtain an image including a depiction of an environmental condition or a transient condition absent from the first panoramic imagery; and transform, using a machine learning model, the first panoramic imagery into second panoramic imagery depicting the environmental condition or the transient condition overlaying the first panoramic imagery. Claim 12 12. A system for providing panoramic imagery, the system comprising: one or more processors; and a non-transitory memory storing instructions that, when executed by the one or more processors, cause the system to: obtain first panoramic imagery depicting a geographic area at a first time, obtain an image depicting at least a portion of the geographic area at a second time later than the first time, the image further depicting one or more features not depicted in the first panoramic imagery, and transform, using a machine learning model, the first panoramic imagery into second panoramic imagery depicting the one or more features and including at least a portion of the first panoramic imagery. Therefore claims 1 and 12 of ‘955 patent teach every limitation recited in claims 1 and 11 respectively of the ‘937 application . Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. Analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added): 1. A method for creating panoramic imagery, the method comprising: (a) obtaining, by one or more processors, first panoramic imagery depicting a geographic area; (b) obtaining, by the one or more processors, an image including a depiction of an environmental condition or a transient condition absent from the first panoramic imagery; and (c) transforming, by the one or more processors and using a machine learning model, the first panoramic imagery into second panoramic imagery depicting the environmental condition or the transient condition overlaying the first panoramic imagery. Step 1 : Claim 1 recites at least one step or act. Thus, the claim is directed to a process, which is one of the statutory categories of invention ( Step 1: YES ). Step 2A Prong One : Claim 1 recites an abstract idea of mental processes. Step (c) is recited at a high level of generality such that it could be practically performed in the human mind. The concept falls into the “mental processes” group of abstract ideas, which includes observation, evaluation, judgment and opinion. The limitation, interpreted under their broadest reasonable interpretation and in consistent with the specification, covers performance of the limitations in the mind or by generic computer components. That is, other than reciting “by the one or more processors” and “using a machine learning model”, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “by the one or more processors” and “using a machine learning model” language, the claim element encompasses a user simply overlaying two images to produce a new image in his/her mind. The mere nominal recitation of a generic processor/machine learning model does not take the claim limitation out of the mental processes grouping. Thus, the claim recites a mental process. See MPEP 2106.04 and the 2019 PEG. ( Step 2A Prong One YES) Step 2A Prong Two : Claim 1 as a whole does not integrate the recited judicial exception into a practical application of the exception. The claim recites the following additional elements: (1) obtaining, first panoramic imagery depicting a geographic area, (2) obtaining, an image including a depiction of an environmental condition or a transient condition absent from the first panoramic imagery, (3) one or more processors and (4) a machine learning model. The two obtaining steps are recited at a high level of generality (i.e., as a general means of gathering data for use in step (c)), and amount to mere data gathering, which is a form of insignificant extra-solution activity. The processor and the machine learning model are recited at a high level of generality and merely add generic computer components to perform the act and therefore fail to provide an improvement to the technology or technical field. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application. ( Step 2A Prong Two No ). Step 2B : Under the 2019 PEG, a conclusion that an additional element is insignificant extra-solution activity in Step 2A should be reevaluated in Step 2B. Here, the two “obtaining” steps are considered to be extra-solution activity in Step 2A, and thus they are reevaluated in Step 2B to determine if they are more than what is well-understood, routine, conventional activity in the field. The obtaining steps reciting mere collection or gathering of data using a generic processor is a well-understood, routine, and conventional function when it is claimed in a merely generic manner (as it is here). These additional elements, taken individually and in combination, do not result in the claim, as a whole, amounting to significantly more than the abstract idea. See MPEP 2106.05. ( Step 2B: NO ). Claim 1 is not eligible . Claim 2 recites: “The method of claim 1, wherein using the machine learning model includes: applying a generator network of a generative adversarial network (GAN), the GAN including the generator network and a discriminator network, to the first panoramic imagery and the image to: extract the one or more features from the image; and merge the one or more features into the first panoramic imagery to generate the second panoramic imagery”. The “extract” and “merge” acts can be practically performed mentally. Further “a generator network of a generative adversarial network (GAN)” is recited in high level of generality and merely adds generic computer component to perform the act and therefore fails to provide an improvement to the technology or technical field. Therefore the additional elements do not integrate the abstract idea into a practical application. Claim 2 is not eligible. Claim 3 recites: “The method of claim 2, wherein transforming the first panoramic imagery includes: applying the discriminator network to the second panoramic imagery to classify the second panoramic imagery as real or generated; if the discriminator network classifies the second panoramic imagery as generated, applying the generator network to the first panoramic imagery and the image to generate third panoramic imagery depicting the one or more features”. The “classifying” and “generate third panoramic imagery” can be practically performed mentally. Further the recited “generator” and “discriminator” is recited in high level of generality and merely adds generic computer component to perform the acts and therefore fails to provide an improvement to the technology or technical field. Therefore the additional elements do not integrate the abstract idea into a practical application. Claim 3 is not eligible. Claim 4 recites: “The method of claim 2, further comprising: training the generator network using a plurality of training panoramic images and a plurality of training images to extract features from the plurality of training images and merge the features into the plurality of training panoramic images; and training the discriminator network using the plurality of training panoramic images and a plurality of generated panoramic images generated by the generator network to classify the plurality of generated panoramic images as generated or real, wherein training the generator network further includes training the generator network using classifications from the discriminator network”. Claim 4 recites training the generator and discriminator, which involves mathematical calculations. Claim 4 is not eligible. Claim 5 recites: “The method of claim 2, wherein transforming the first panoramic imagery includes: applying the generator network to the first panoramic imagery to insert one or more light conditions indicative of a first amount of daylight into the first panoramic imagery, the first amount of daylight different from a second amount of daylight depicted in the first panoramic imagery”. The “applying… to insert” step can be performed practically mentally. For example, a user can overlay a brighter background image onto the first panoramic imagery to generate a second panoramic imagery. The “generator” is recited in high level of generality and merely adds generic computer component to perform the acts and therefore fails to provide an improvement to the technology or technical field. Therefore the additional elements do not integrate the abstract idea into a practical application. Claim 5 is not eligible. Claim 6 recites: “The method of claim 5, further comprising: training the generator network using a plurality of training panoramic images and a plurality of training images to identify the one or more light conditions and to insert the one or more light conditions into the first panoramic imagery”. The “training” step recites mathematical calculations. Claim 6 is not eligible. Claims 7-9 recites the details of the environmental condition or the transient condition, i.e., a high crowd level, one of rain, snow, fog or ice, and one of winter, spring, summer, or fall. These additional elements, when combined with the elements recited in claim 1, does not resolve the issue that claim 1 recites a mental process. Further, these additional elements do not integrate the abstract idea recited in claim 1 into a practical application. Claims 7-9 are not eligible Claim 10 r ecites: “The method of claim 1, wherein the image is a non-panoramic image, and wherein transforming the first panoramic imagery includes: extracting the one or more features from the non-panoramic image; identifying a projection type of the first panoramic imagery; mapping the one or more physical objects to a coordinate system of the projection type; and merging the mapped one or more features into the first panoramic imagery to generate the second panoramic imagery”. The recited “extracting”, “identifying”, “mapping” and “merging” steps recite mathematical calculations. Claim 10 is not eligible. Claim 11 is the corresponding apparatus claim of claim 1. Claim 11 recites similar steps as in claim 1. Therefore the recited “transform” step recites mental process. The additional elements include “obtain first panoramic imagery”, “obtain an image”, “one or more processors”, “a non-transitory memory” and “a machine learning model”. The two obtaining steps recite mere data collection and gathering, which is insignificant extra-solution activity that is no more than what is well-understood, routine, conventional activity in the field. The processor, memory and the machine learning model are recited at a high level of generality and merely add generic computer components to perform the act and therefore fail to provide an improvement to the technology or technical field. Accordingly, even in combination, these additional elements neither integrate the abstract idea into a practical application, nor amount to significantly more than the abstract idea. Claim 11 is not eligible. Similar analysis applied to claims 2-10 is applicable to claims 12-20 respectively. Claims 12-20 are not eligible. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-21-aia AIA Claim s 1-6, 8-16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over JAIPURIA et al. (US 20210004608 A1, hereafter JAIPURIA), in view of in view of Yi et al. (US Publication 2018/0240223 A1, hereafter Yi) . As per claim 1 , JAIPURIA teaches a method (Abstract; para. [0011]) for creating panoramic imagery, the method comprising: obtaining, by one or more processors (Abstract; FIG. 1)), first panoramic imagery depicting a geographic area (FIG. 2; para. [0025] “A computing device 115 in a vehicle 110 can acquire a red, green, and blue (RGB) color video images of traffic scene 200 corresponding to an environment around the vehicle 110 using a sensor 116 included in vehicle 110 ”); obtaining, by the one or more processors, an image including a depiction of an environmental condition or a transient condition absent from the first panoramic imagery (JAIPURIA teaches a method for generating a plurality of domain adapted synthetic images by processing the synthetic image with a variational auto encoder-generative adversarial network (VAE-GAN) (para. [0011]). The synthetic image can be adapted from a first domain to a second domain by modifying the synthetic image to change weather and lighting conditions in the synthetic image, for example from a sunny image domain to a rainy image domain (para. [0011], [0030]). For example, traffic scene 200 in FIG. 2 corresponds to a sunny, summer, daytime domain, while traffic scene 300 in FIG. 3 includes rain 310 , illustrated by dashed lines, and therefore corresponds to a rain precipitation domain. JAIPURIA further teaches that during training, VAE-GAN 400 inputs a real world video images 402 of traffic scenes corresponding to both a first domain and a second domain. For example, the training dataset can include real world video images corresponding to day domains and night domains (para. [0031]; FIG. 2-3); The first domain image corresponding to “first imagery” and second domain image corresponds to an image including a depiction of an environmental condition or a transient condition absent from the first panoramic imagery); and transforming, by the one or more processors and using a machine learning model (FIG. 4 showing a VAE-GAN 400 neural network; para. [0031]), the first panoramic imagery into second panoramic imagery depicting the environmental condition or the transient condition overlaying the first panoramic imagery (para. [0031] “… to transform the hidden variables 406 into an output video image 410 that corresponds to the input video image 402 transformed into a user selected domain. For example, a VAE-GAN 400 neural network can be trained to output photorealistic synthetic video images corresponding to a rain domain based on input photorealistic synthetic video images corresponding to a sunny domain”; para. [0032] “… to train VAE-GAN 400 to output video images 410 that match input video images 402 belonging to one selected domain. For example, in this fashion VAE-GAN 400 can be trained to output video images 410 corresponding to one selected domain (winter, rain, night, etc.) despite input video images 402 corresponding to another domain (summer, sunny, day, etc.)”). JAIPURIA, however, does not teach that the first imagery and second imagery are panoramic imageries. Yi discloses a three dimensional image fusion method (Abstract). Specifically, a two dimensional image is projected onto a spherical image (i.e., a panoramic image, see para. [0030]) and a composite spherical image is generated (Abstract; FIG. 2; para. [0029]-[0038]). Taking the combined teachings of JAIPURIA and Yi as a whole, it would have been obvious for a person with ordinary skill in the art before the effective filing date of the claimed invention to consider transforming a first panoramic imagery into a composite panoramic imagery as performed by Yi. Doing so would bring a more vivid visual experience (3D) to people as recognized by Yi (para. [0003]). As per claim 2 , dependent upon claim 1, JAIPURIA in view of Yi teaches wherein using the machine learning model includes: applying a generator network of a generative adversarial network (GAN), the GAN including the generator network and a discriminator network, to the first panoramic imagery and the image to: extract the one or more features from the image; and merge the one or more features into the first panoramic imagery to generate the second panoramic imagery (JAIPURIA FIG. 4 shows a VAE-GAN 400 neural network which includes an encoder 404, a generator 408 and a discriminator 412. The encoder 404 includes a plurality of convolutional layers that encodes an input video image 402 into hidden variables 406 that are assumed to have Gaussian distributions. The hidden variables 406 are input to generator 408 which uses a plurality of convolutional layers programmed to transform the hidden variables 406 into an output video image 410 that corresponds to the input video image 402 transformed into a user selected domain. Therefore the generator extracts the one or more features from the image in the second domain, and merge the one or more features into the image in the first domain to generate the image in the second domain (#410)). As per claim 3 , dependent upon claim 2, JAIPURIA in view of Yi teaches wherein transforming the first panoramic imagery includes: applying the discriminator network to the second panoramic imagery to classify the second panoramic imagery as real or generated (JAIPURIA FIG. 4 the output image in the second domain (#410)) is input into the discriminator #412 to determine whether #410 is real or generated (para. [0032])); if the discriminator network classifies the second panoramic imagery as generated, applying the generator network to the first panoramic imagery and the image to generate third panoramic imagery depicting the one or more features (JAIPURIA FIG. 4; para. [0032] “This loss function 414 can be back propagated to the encoder 404 /generator 408 section of VAE-GAN 400 at training time to train VAE-GAN 400 to output video images 410 that match input video images 402 belonging to one selected domain”. That says the loss function 414 can be back propagated to the encoder 404 /generator 408 to start a new iteration to generate a new output #410). As per claim 4 , dependent upon claim 2, JAIPURIA in view of Yi further teaches: training the generator network using a plurality of training panoramic images and a plurality of training images to extract features from the plurality of training images and merge the features into the plurality of training panoramic images (See below); and training the discriminator network using the plurality of training panoramic images and a plurality of generated panoramic images generated by the generator network to classify the plurality of generated panoramic images as generated or real, wherein training the generator network further includes training the generator network using classifications from the discriminator network (JAIPURIA FIG. 4 and para. [0031]-[0032] describes the training of the generator and discriminator, including the following highlights: “A training time, VAE-GAN 400 inputs a real world video images 402 of traffic scenes corresponding to both a first domain and a second domain”, “The hidden variables 406 are input to generator 408 which uses a plurality of convolutional layers programmed to transform the hidden variables 406 into an output video image 410 that corresponds to the input video image 402 transformed into a user selected domain”, “The loss function 414 is derived from a comparison of the output video image 410 to probability distributions derived from real world image data to determine whether the output video image 410 is similar to real world image data despite not having a real world image that exactly corresponds to the output video image 410 ”, “This loss function 414 can be back propagated to the encoder 404 /generator 408 section of VAE-GAN 400 at training time to train VAE-GAN 400 to output video images 410 that match input video images 402 belonging to one selected domain”). As per claim 5 , dependent upon claim 2, JAIPURIA in view of Yi further teaches transforming the first panoramic imagery includes: applying the generator network to the first panoramic imagery to insert one or more light conditions indicative of a first amount of daylight into the first panoramic imagery, the first amount of daylight different from a second amount of daylight depicted in the first panoramic imagery (JAIPURIA para. [0010] “Adding domain data to a synthetic image can include modifying the synthetic image to simulate the effects of different environmental conditions or noise factors such as precipitation including rain or snow, atmospheric/lighting conditions including fog or night, and seasonal conditions including winter and spring”; para. [0011] “The synthetic image can be adapted from a first domain to a second domain by modifying the synthetic image to change weather and lighting conditions in the synthetic image”; para. [0031] “For example, the training dataset can include real world video images corresponding to day domains and night domains”; para. [0033] “For example, VAE-GAN 500 can be trained to input synthetic video images 502 corresponding to a sunny summer day domain and output synthetic video images 510 corresponding to a winter night domain”). As per claim 6 , dependent upon claim 5, JAIPURIA in view of Yi further teaches: training the generator network using a plurality of training panoramic images and a plurality of training images to identify the one or more light conditions and to insert the one or more light conditions into the first panoramic imagery (JAIPURIA para. [0010] “Adding domain data to a synthetic image can include modifying the synthetic image to simulate the effects of different environmental conditions or noise factors such as precipitation including rain or snow, atmospheric/lighting conditions including fog or night, and seasonal conditions including winter and spring”; para. [0011] “The synthetic image can be adapted from a first domain to a second domain by modifying the synthetic image to change weather and lighting conditions in the synthetic image”; para. [0031] “A training time, VAE-GAN 400 inputs a real world video images 402 of traffic scenes corresponding to both a first domain and a second domain. For example, the training dataset can include real world video images corresponding to day domains and night domains”; para. [0032] “For example, in this fashion VAE-GAN 400 can be trained to output video images 410 corresponding to one selected domain (winter, rain, night, etc.) despite input video images 402 corresponding to another domain (summer, sunny, day, etc.)”). As per claim 8 , dependent upon claim 1, JAIPURIA in view of Yi further teaches the environmental condition or the transient condition corresponds to one of rain, snow, fog, or ice (JAIPURIA para. [0010] “Adding domain data to a synthetic image can include modifying the synthetic image to simulate the effects of different environmental conditions or noise factors such as precipitation including rain or snow, atmospheric/lighting conditions including fog or night, and seasonal conditions including winter and spring”; para. [0031] “For example, a VAE-GAN 400 neural network can be trained to output photorealistic synthetic video images corresponding to a rain domain based on input photorealistic synthetic video images corresponding to a sunny domain”). As per claim 9 , dependent upon claim 1, JAIPURIA in view of Yi further teaches the environmental condition or the transient condition is one of winter, spring, summer, or fall (JAIPURIA para. [0010] “Adding domain data to a synthetic image can include modifying the synthetic image to simulate the effects of different environmental conditions or noise factors such as precipitation including rain or snow, atmospheric/lighting conditions including fog or night, and seasonal conditions including winter and spring”; para. [0032] “For example, in this fashion VAE-GAN 400 can be trained to output video images 410 corresponding to one selected domain (winter, rain, night, etc.) despite input video images 402 corresponding to another domain (summer, sunny, day, etc.)”). As per claim 10 , dependent upon claim 1, JAIPURIA in view of Yi teaches the image is a non-panoramic image (JAIPURIA FIG. 2-3; para. [0025]: “A computing device 115 in a vehicle 110 can acquire a red, green, and blue (RGB) color video images of traffic scene 200 corresponding to an environment around the vehicle 110 using a sensor 116 included in vehicle 110 ”), and wherein transforming the first panoramic imagery includes: extracting the one or more features from the non-panoramic image (JAIPURIA FIG. 4; para. [0031] “The encoder 404 includes a plurality of convolutional layers that encodes an input video image 402 into hidden variables 406 that are assumed to have Gaussian distributions … The hidden variables 406 are input to generator 408 which uses a plurality of convolutional layers programmed to transform the hidden variables 406 into an output video image 410 that corresponds to the input video image 402 transformed into a user selected domain”); and merging the extracted one or more features into the first panoramic imagery to generate the second panoramic imagery (JAIPURIA FIG. 4; para. [0010] “Adding domain data to a synthetic image can include modifying the synthetic image to simulate the effects of different environmental conditions or noise factors such as precipitation including rain or snow, atmospheric/lighting conditions including fog or night, and seasonal conditions including winter and spring”; para. [0030] “A VAE-GAN is a neural network configured to input a synthetic video image rendered in a first image domain and output a video image modified to appear as an accurate photorealistic video image rendered in a second domain”: para. [0031] “The hidden variables 406 are input to generator 408 which uses a plurality of convolutional layers programmed to transform the hidden variables 406 into an output video image 410 that corresponds to the input video image 402 transformed into a user selected domain”). See motivation and rationale applied to claim 1 of combining JAIPURIA with Yi for processing panoramic imagery. JAIPURIA, however, does not further teach: identifying a projection type of the first panoramic imagery; and mapping the one or more physical objects to a coordinate system of the projection type. Yi in an analogous field discloses a three dimensional image fusion method (Abstract). Specifically, a two dimensional image is projected onto a spherical image (i.e., a panoramic image, see para. [0030]) and a composite spherical image is generated (Abstract; FIG. 2; para. [0029]-[0038]). Specifically, Yi teaches: identifying a projection type of the first panoramic imagery (Yi Abstract); mapping one or more features to a coordinate system of the projection type (Yi FIG. 2; FIG. 4; para. [0029]-[0038]); and merging the mapped one or more features into the first panoramic imagery to generate the second panoramic imagery (Yi FIG. 2 S204). It would have been obvious for a person with ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of JAIPURIA to incorporate the teaching of Yi to identify a projection type of the first panoramic imagery and map one or more features to a coordinate system of the projection type. Doing so would allow image features in 2D coordinate system can be seamlessly fused into to a 3D spherical image (panoramic imagery) as recognized by Yi (para. [0009]). As per claim 11 , JAIPURIA in view of Yi teaches a system (JAIPURIA Abstract; FIG. 1) for providing panoramic imagery, the system comprising: one or more processors (JAIPURIA Abstract; FIG. 1); and a non-transitory memory storing instructions that, when executed by the one or more processors (JAIPURIA Abstract; FIG. 1), cause the system to: obtain first panoramic imagery depicting a geographic area (See below); obtain an image including a depiction of an environmental condition or a transient condition absent from the first panoramic imagery (See below); and transform, using a machine learning model, the first panoramic imagery into second panoramic imagery depicting the environmental condition or the transient condition overlaying the first panoramic imagery (Claim 11 recites steps corresponding to the steps recited in method claim 1. Therefore, the recited steps of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding steps in its corresponding method claim 1. Additionally, the motivation and rationale for combining JAIPURIA and Yi applied in claim 1 is applicable to claim 11). Regarding claim 12, claim 12 recites a system with elements corresponding to the elements recited in claim 2. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 2. Regarding claim 13, claim 13 recites a system with elements corresponding to the elements recited in claim 3. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 3. Regarding claim 14, claim 14 recites a system with elements corresponding to the elements recited in claim 4. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 4. Regarding claim 15, claim 15 recites a system with elements corresponding to the elements recited in claim 5. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 5. Regarding claim 16, claim 16 recites a system with elements corresponding to the elements recited in claim 6. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 6. Regarding claim 18, claim 18 recites a system with elements corresponding to the elements recited in claim 8. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 8. Regarding claim 19, claim 19 recites a system with elements corresponding to the elements recited in claim 9. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 9. Regarding claim 20, claim 20 recites a system with elements corresponding to the elements recited in claim 10. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi in the same manner as the corresponding elements in its corresponding method claim, claim 10. Additionally, the motivation and rationale for combining JAIPURIA and Yi applied in claim 10 is applicable to claim 20 . 07-21-aia AIA Claim s 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over JAIPURIA et al. (US 20210004608 A1, hereafter JAIPURIA), in view of in view of Yi et al. (US Publication 2018/0240223 A1, hereafter Yi), as applied above to claims 1 and 11 respectively, and further in view of Mohandoss et al. (US 20210158570 A1, hereafter Mohandoss) . As per claim 7 , dependent upon claim 1, JAIPURIA in view of Yi teaches wherein the environmental condition or the transient condition includes pedestrian (JAIPURIA para. [0012] “The vehicle can be operated based on determining a vehicle path based on the objects and locations. The moving objects can include one or more of a pedestrian and a vehicle. The VAE-GAN can be trained by constraining hidden variables to be consistent with a desired output domain”). JAIPURIA in view of Yi, however, does not teach the environmental condition or the transient condition corresponds to a high crowd level. Mohandoss in the same field of endeavor discloses a method of training generative adversarial networks to shot-match two unmatched images in a context-sensitive manner (Abstract). The method includes accessing a trained generative adversarial network including a trained generator model and a trained discriminator model. A source image and a reference image may be inputted into the generator model to generate a modified source image (Abstract). As shown in FIG. 6 second row, the source image on the right can be modified to match the color in the reference image on the left, the reference image on the left having a high crowd level. See FIG. 6-7 and descriptions in para. [0059]-[0062]. It would have been obvious for a person with ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of JAIPURIA and Yi to incorporate the teaching of Mohandoss to include an environmental condition or transient condition corresponding to a high crowd level. Doing so would match the color information in an source image with the color information in a reference image with high crowed level in a context-sensitive manner (Mohandoss para. [0018]). Regarding claim 17, claim 17 recites a system with elements corresponding to the elements recited in claim 7. Therefore, the recited elements of this claim are mapped to JAIPURIA in view of Yi and Mohandoss in the same manner as the corresponding elements in its corresponding method claim, claim 7. Additionally, the motivation and rationale for combining JAIPURIA, Yi and Mohandoss applied in claim 7 is applicable to claim 17 . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and is recorded in form PTO-892 . Shechtman teaches a method (Abstract) for creating panoramic imagery, the method comprising: obtaining first imagery depicting a geographic area (FIG. 1 “Background Image 106”); obtaining an image depicting one or more physical objects absent from the first imagery (FIG. 1 “Foreground Object 108”); and transforming, using a machine learning model first imagery into second imagery depicting the one or more physical objects and including at least a portion of the first imagery (FIG. 1 “Composite Image 114”; para. [0045]-[0048]). Contact Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUEMEI G CHEN whose telephone number is (571)270-3480. The examiner can normally be reached Monday-Friday 9am-6pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /XUEMEI G CHEN/Primary Examiner, Art Unit 2661 Application/Control Number: 18/781,937 Page 2 Art Unit: 2661 Application/Control Number: 18/781,937 Page 3 Art Unit: 2661 Application/Control Number: 18/781,937 Page 4 Art Unit: 2661 Application/Control Number: 18/781,937 Page 5 Art Unit: 2661 Application/Control Number: 18/781,937 Page 6 Art Unit: 2661 Application/Control Number: 18/781,937 Page 7 Art Unit: 2661 Application/Control Number: 18/781,937 Page 8 Art Unit: 2661 Application/Control Number: 18/781,937 Page 9 Art Unit: 2661 Application/Control Number: 18/781,937 Page 10 Art Unit: 2661 Application/Control Number: 18/781,937 Page 11 Art Unit: 2661 Application/Control Number: 18/781,937 Page 12 Art Unit: 2661 Application/Control Number: 18/781,937 Page 13 Art Unit: 2661 Application/Control Number: 18/781,937 Page 14 Art Unit: 2661 Application/Control Number: 18/781,937 Page 15 Art Unit: 2661 Application/Control Number: 18/781,937 Page 16 Art Unit: 2661 Application/Control Number: 18/781,937 Page 17 Art Unit: 2661 Application/Control Number: 18/781,937 Page 18 Art Unit: 2661 Application/Control Number: 18/781,937 Page 19 Art Unit: 2661 Application/Control Number: 18/781,937 Page 20 Art Unit: 2661 Application/Control Number: 18/781,937 Page 21 Art Unit: 2661 Application/Control Number: 18/781,937 Page 22 Art Unit: 2661 Application/Control Number: 18/781,937 Page 23 Art Unit: 2661 Application/Control Number: 18/781,937 Page 24 Art Unit: 2661 Application/Control Number: 18/781,937 Page 25 Art Unit: 2661 Application/Control Number: 18/781,937 Page 26 Art Unit: 2661 Application/Control Number: 18/781,937 Page 27 Art Unit: 2661 Application/Control Number: 18/781,937 Page 28 Art Unit: 2661 Application/Control Number: 18/781,937 Page 29 Art Unit: 2661 Application/Control Number: 18/781,937 Page 30 Art Unit: 2661
Prosecution Insights