DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, see remarks, filed 06/08/2026, with respect to claims 1-5, 8-10, and 16-27 have been fully considered, but are not persuasive. The applicant states on page 7, “the combination fails to teach or suggest “using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models, selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image, and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model.” Page 3 of the Office Action concedes that Bjorn fails to describe or suggest the above features. Instead page 4 of the Office Action asserts that Endoh teaches the above features.”
The office would like to bring to applicant’s attention that as stated on Pages 3-4 of the office action, BJORN et al. (GB 2542118 A), hereinafter referenced as BJORN, is said to explicitly teach and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model (Fig. 3, Page 5, Lines [10-12] – BJORN discloses at step S316, an image from the first set of images a spatially corresponding image selected from the transformed image set are provided as first and second channel inputs to a two-channel CNN. Page 5, Lines [26-28] – BJORN further discloses at step S318, one or both of the images that were used as the first and second channel inputs is classified as either being associated with a change to the structure or not being associated with a change. See also Page 7, Lines [21-28].).
BJORN fails to explicitly teach using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image;
However, ENDOH et al. (US 20190303766 A1), hereinafter referenced as ENDOH, explicitly teaches using the second image as input to a set of machine learning, ML, models (Fig. 6, Paragraph [0071] – ENDOH discloses the modules 10A to 10C each include the learning model of the DL and the autoencoder. As another aspect, at the time of recognition, test data [wherein test data is the second image] is input to the modules 10A to 10C as input data.)
to generate a reconstructed image of the second image from each of the set of ML models (Fig. 6, Paragraph [0071] – ENDOH discloses the reconstruction error output from the autoencoder of each of the modules 10A to 10C is output to the selection unit 11, and the output result of the learning model of the DL of each of the modules 10A to 10C is output to the output unit 12. Paragraph [0101] – ENDOH further discloses to the AE, set is a layer structure of the model including the encoder corresponding to the NN that compresses the learning data into feature expression the dimension of which is reduced from the dimension of the learning data, and the decoder corresponding to the NN that receives the output from the encoder as an input and reconstructs original learning data from the feature expression.);
selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image (Fig. 6, Paragraph [0072] – ENDOH discloses the selection unit 11 selects, from among the output results of the DL output from each of the modules 10A to 10C, the output result of the DL of the module 10 in which the reconstruction error of the autoencoder output from each of the modules 10A to 10C is the smallest.);
The applicant further argues on page 7 that “Endoh only discloses that input data is fed through multiple modules with different autoencoders. A selection unit selects the module with the lowest reconstruction error to select the output from the respective module. (See Endoh, para. [0071]-[0072]). In other words, Endoh is attempting to select the best autoencoder to reconstruct an image to provide as input. (See Endoh, para. [0078]-[0093], Figs. 8-10). Notably, Endoh does not perform the additional step of feeding two input images back into the selected module to detect if there are any changes in the physical environment between the two images.”
The office would like to bring to applicant’s attention that the cited prior art said to teach the above stated limitation of “detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model” is BJORN, not ENDOH. As stated on Pages 3-4 of the office action, BJORN is said to explicitly teach and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model (Fig. 3, Page 5, Lines [10-12] – BJORN discloses at step S316, an image from the first set of images a spatially corresponding image selected from the transformed image set are provided as first and second channel inputs to a two-channel CNN. Page 5, Lines [26-28] – BJORN further discloses at step S318, one or both of the images that were used as the first and second channel inputs is classified as either being associated with a change to the structure or not being associated with a change. See also Page 7, Lines [21-28].).
The applicant further argues on pages 7-8 that “Bjorn and Endoh cannot be meaningfully combined. It is not clear how Bjorn could be modified to use the methods disclosed by Endoh to determine changes in a structure at two different time periods. Thus, attempting to modify Bjorn with Endoh would render Bjorn inoperable for its intended purpose.”
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, the Office has clearly described that it would have been obvious to one of ordinary skill in the art at the time the invention was made to modify the teachings of BJORN in view of ENDOH, which are both related to machine learning models for implementing recognition using input data that provide an enhanced method for reliably and accurately detecting changes from input data, wherein having BJORN’s method for detecting changes in a physical environment with minimal manual training effort and enhancing adaptability to provide improved accuracy. The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and ENDOH relate to machine learning models for implementing recognition using input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and ENDOH relates to an adaptability calculation method, an adaptability calculation device, a computer-readable recording medium, an identification method, and an identification device with respect to input data, wherein an output from a wrong learning model is prevented from being selected, so that a possibility of failing in recognition can be reduced. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and ENDOH (US 20190303766 A1), Paragraph [0002, 0067].
In this case, claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over BJORN (GB 2542118 A), hereinafter referenced as BJORN in view of ENDOH (US 20190303766 A1), hereinafter referenced as ENDOH.
Regarding claim 1, BJORN teaches a method for detecting changes in a physical environment (Fig. 3, Page 9, Lines [26-28] – BJORN discloses the approach described herein is to detect change using a two-channel CNN. The network accepts pairs of approximately registered image patches taken at different times and classifies them to detect anomalous changes.),
the method performed by an apparatus (Fig. 1, Page 3, Lines [23-32] - BJORN discloses #122 called the computer and #112 called image capture device) and comprising:
obtaining a first image representing the physical environment at a first time instance (Fig. 3, Page 4, Lines [27-29] – BJORN discloses step S312, the first set of images [wherein the first set of images comprises a first image] are processed relative to one another in chunks associated with sections of the traversal of the image capture device 112 within the structure.);
obtaining a second image representing the physical environment at a second time instance (Fig. 3, Page 4, Lines [27-29] – BJORN discloses at step S312, the first set of images are processed relative to one another in chunks associated with sections of the traversal of the image capture device 112 within the structure. In particular, structure from motion analysis is used to return point clouds and camera pose estimations. The same is also done for the second set of images [wherein the second set of images comprises a second image].);
Although BJORN further teaches and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model (Fig. 3, Page 5, Lines [10-12] – BJORN discloses at step S316, an image from the first set of images a spatially corresponding image selected from the transformed image set are provided as first and second channel inputs to a two-channel CNN. Page 5, Lines [26-28] – BJORN further discloses at step S318, one or both of the images that were used as the first and second channel inputs is classified as either being associated with a change to the structure or not being associated with a change. See also Page 7, Lines [21-28].).
BJORN fails to explicitly teach using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image;
However, ENDOH explicitly teaches using the second image as input to a set of machine learning, ML, models (Fig. 6, Paragraph [0071] – ENDOH discloses the modules 10A to 10C each include the learning model of the DL and the autoencoder. As another aspect, at the time of recognition, test data [wherein test data is the second image] is input to the modules 10A to 10C as input data.)
to generate a reconstructed image of the second image from each of the set of ML models (Fig. 6, Paragraph [0071] – ENDOH discloses the reconstruction error output from the autoencoder of each of the modules 10A to 10C is output to the selection unit 11, and the output result of the learning model of the DL of each of the modules 10A to 10C is output to the output unit 12. Paragraph [0101] – ENDOH further discloses to the AE, set is a layer structure of the model including the encoder corresponding to the NN that compresses the learning data into feature expression the dimension of which is reduced from the dimension of the learning data, and the decoder corresponding to the NN that receives the output from the encoder as an input and reconstructs original learning data from the feature expression.);
selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image (Fig. 6, Paragraph [0072] – ENDOH discloses the selection unit 11 selects, from among the output results of the DL output from each of the modules 10A to 10C, the output result of the DL of the module 10 in which the reconstruction error of the autoencoder output from each of the modules 10A to 10C is the smallest.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model, with the teachings of ENDOH of having using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image.
Wherein BJORN’s method for detecting changes in a physical environment wherein having using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image.
The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and ENDOH relate to machine learning models for implementing recognition using input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and ENDOH relates to an adaptability calculation method, an adaptability calculation device, a computer-readable recording medium, an identification method, and an identification device with respect to input data, wherein an output from a wrong learning model is prevented from being selected, so that a possibility of failing in recognition can be reduced. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and ENDOH (US 20190303766 A1), Paragraph [0002, 0067].
The applicant further argues on page 8, “For at least the reasons discussed above, the Applicant respectfully submits that Claim 1 is patentable over the cited art. Claim 16 recite similar recitations as those discussed above with reference to Claim 1. For similar reasons, Applicant respectfully submits that Claim 16 is also patentable over the cited references. Accordingly, Applicant respectfully requests the rejection of Claims 1 and 16 under 35 U.S.C. §103 to be withdrawn and the claims allowed. The remaining dependent claims are patentable at least per the patentability of the independent claims from which they depend.” Applicant further states on page 9, “For at least the reasons discussed above, the Applicant respectfully submits that all claims are patentable. Accordingly, a Notice of Allowance is respectfully requested in due course.”
In response, the office does not find this argument persuasive based on the same reasons set forth above and the rejection below. The office respectfully encourages the applicant to amend the claims to overcome the prior arts of record.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed
invention is not identically disclosed as set forth in section 102 of this title, if the
differences between the claimed invention and the prior art are such that the claimed
invention as a whole would have been obvious before the effective filing date of the
claimed invention to a person having ordinary skill in the art to which the claimed
invention pertains. Patentability shall not be negated by the manner in which the
invention was made.
Claims 1, 3, 16, 18, 26 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over BJORN (GB 2542118 A), hereinafter referenced as BJORN in view of ENDOH (US 20190303766 A1), hereinafter referenced as ENDOH.
Regarding claim 1, BJORN teaches a method for detecting changes in a physical environment (Fig. 3, Page 9, Lines [26-28] – BJORN discloses the approach described herein is to detect change using a two-channel CNN. The network accepts pairs of approximately registered image patches taken at different times and classifies them to detect anomalous changes.),
the method performed by an apparatus (Fig. 1, Page 3, Lines [23-32] - BJORN discloses #122 called the computer and #112 called image capture device) and comprising:
obtaining a first image representing the physical environment at a first time instance (Fig. 3, Page 4, Lines [27-29] – BJORN discloses step S312, the first set of images [wherein the first set of images comprises a first image] are processed relative to one another in chunks associated with sections of the traversal of the image capture device 112 within the structure.);
obtaining a second image representing the physical environment at a second time instance (Fig. 3, Page 4, Lines [27-29] – BJORN discloses at step S312, the first set of images are processed relative to one another in chunks associated with sections of the traversal of the image capture device 112 within the structure. In particular, structure from motion analysis is used to return point clouds and camera pose estimations. The same is also done for the second set of images [wherein the second set of images comprises a second image].);
Although BJORN further teaches and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model (Fig. 3, Page 5, Lines [10-12] – BJORN discloses at step S316, an image from the first set of images a spatially corresponding image selected from the transformed image set are provided as first and second channel inputs to a two-channel CNN. Page 5, Lines [26-28] – BJORN further discloses at step S318, one or both of the images that were used as the first and second channel inputs is classified as either being associated with a change to the structure or not being associated with a change. See also Page 7, Lines [21-28].).
BJORN fails to explicitly teach using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image;
However, ENDOH explicitly teaches using the second image as input to a set of machine learning, ML, models (Fig. 6, Paragraph [0071] – ENDOH discloses the modules 10A to 10C each include the learning model of the DL and the autoencoder. As another aspect, at the time of recognition, test data [wherein test data is the second image] is input to the modules 10A to 10C as input data.)
to generate a reconstructed image of the second image from each of the set of ML models (Fig. 6, Paragraph [0071] – ENDOH discloses the reconstruction error output from the autoencoder of each of the modules 10A to 10C is output to the selection unit 11, and the output result of the learning model of the DL of each of the modules 10A to 10C is output to the output unit 12. Paragraph [0101] – ENDOH further discloses to the AE, set is a layer structure of the model including the encoder corresponding to the NN that compresses the learning data into feature expression the dimension of which is reduced from the dimension of the learning data, and the decoder corresponding to the NN that receives the output from the encoder as an input and reconstructs original learning data from the feature expression.);
selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image (Fig. 6, Paragraph [0072] – ENDOH discloses the selection unit 11 selects, from among the output results of the DL output from each of the modules 10A to 10C, the output result of the DL of the module 10 in which the reconstruction error of the autoencoder output from each of the modules 10A to 10C is the smallest.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model, with the teachings of ENDOH of having using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image.
Wherein BJORN’s method for detecting changes in a physical environment wherein having using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image.
The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and ENDOH relate to machine learning models for implementing recognition using input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and ENDOH relates to an adaptability calculation method, an adaptability calculation device, a computer-readable recording medium, an identification method, and an identification device with respect to input data, wherein an output from a wrong learning model is prevented from being selected, so that a possibility of failing in recognition can be reduced. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and ENDOH (US 20190303766 A1), Paragraph [0002, 0067].
Regarding claim 3, BJORN in view of ENDOH teach the method of claim 1,
BJORN further teaches wherein the first image is among a plurality of images most similar to the second image (Fig. 3, Page 5, Lines [10-15] – BJORN discloses an image from the first set of images a spatially corresponding image selected from the transformed image set are provided as first and second channel inputs to a two-channel CNN. In order to select a spatially corresponding image, an image is searched for in the transformed image set that overlaps the image from the first set of images, optionally the search may look for an image that has the greatest overlap with the image from the first set of images. See also Page 7, Lines [16-25].).
Regarding claim 16, BJORN teaches an apparatus for detecting changes in a physical environment (Fig. 1, Page 3, Lines [1-2, 9-12] – BJORN discloses Figure 1 shows a cross-section through a tunnel lining 110 in which an example image capture device 112 is positioned. The image capture device 112 further comprises a memory and communication module 120 that is arranged to record the captured images and subsequently communicate them wirelessly to a computer 122 [wherein an apparatus is comprised of image capture device 112 and computer 122].),
the apparatus comprising a processing circuitry causing the apparatus to be operative (Fig. 2, Page 3, Lines [13-14] – BJORN discloses computer 122 comprises a micro-processor 210 arranged to execute computer readable instructions as may be provided to the computer 122) to:
obtain a first image representing the physical environment at a first time instance (Fig. 3, Page 4, Lines [27-29] – BJORN discloses step S312, the first set of images [wherein the first set of images comprises a first image] are processed relative to one another in chunks associated with sections of the traversal of the image capture device 112 within the structure.);
obtain a second image representing the physical environment at a second time instance (Fig. 3, Page 4, Lines [27-29] – BJORN discloses at step S312, the first set of images are processed relative to one another in chunks associated with sections of the traversal of the image capture device 112 within the structure. In particular, structure from motion analysis is used to return point clouds and camera pose estimations. The same is also done for the second set of images [wherein the second set of images comprises a second image].);
Although BJORN further teaches and detect if there are changes in the physical environment by using the first image and the second image as input to the selected ML model (Fig. 3, Page 5, Lines [10-12] – BJORN discloses at step S316, an image from the first set of images a spatially corresponding image selected from the transformed image set are provided as first and second channel inputs to a two-channel CNN. Page 5, Lines [26-28] – BJORN further discloses at step S318, one or both of the images that were used as the first and second channel inputs is classified as either being associated with a change to the structure or not being associated with a change. See also Page 7, Lines [21-28].).
BJORN fails to explicitly teach use the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; select an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image;
However, ENDOH explicitly teaches use the second image as input to a set of machine learning, ML, models (Fig. 6, Paragraph [0071] – ENDOH discloses the modules 10A to 10C each include the learning model of the DL and the autoencoder. As another aspect, at the time of recognition, test data [wherein test data is the second image] is input to the modules 10A to 10C as input data.)
to generate a reconstructed image of the second image from each of the set of ML models (Fig. 6, Paragraph [0071] – ENDOH discloses the reconstruction error output from the autoencoder of each of the modules 10A to 10C is output to the selection unit 11, and the output result of the learning model of the DL of each of the modules 10A to 10C is output to the output unit 12. Paragraph [0101] – ENDOH further discloses to the AE, set is a layer structure of the model including the encoder corresponding to the NN that compresses the learning data into feature expression the dimension of which is reduced from the dimension of the learning data, and the decoder corresponding to the NN that receives the output from the encoder as an input and reconstructs original learning data from the feature expression.);
select an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image (Fig. 6, Paragraph [0072] – ENDOH discloses the selection unit 11 selects, from among the output results of the DL output from each of the modules 10A to 10C, the output result of the DL of the module 10 in which the reconstruction error of the autoencoder output from each of the modules 10A to 10C is the smallest.);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; and detect if there are changes in the physical environment by using the first image and the second image as input to the selected ML model, with the teachings of ENDOH of having use the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; select an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image;
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein having use the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; select an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image;
The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and ENDOH relate to machine learning models for implementing recognition using input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and ENDOH relates to an adaptability calculation method, an adaptability calculation device, a computer-readable recording medium, an identification method, and an identification device with respect to input data, wherein an output from a wrong learning model is prevented from being selected, so that a possibility of failing in recognition can be reduced. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and ENDOH (US 20190303766 A1), Paragraph [0002, 0067].
Regarding claim 18, BJORN in view of ENDOH teach the apparatus of claim 16,
BJORN further teaches wherein the first image is among a plurality of images most similar to the second image (Fig. 3, Page 5, Lines [10-15] – BJORN discloses an image from the first set of images a spatially corresponding image selected from the transformed image set are provided as first and second channel inputs to a two-channel CNN. In order to select a spatially corresponding image, an image is searched for in the transformed image set that overlaps the image from the first set of images, optionally the search may look for an image that has the greatest overlap with the image from the first set of images. See also Page 7, Lines [16-25].).
Regarding claim 26, BJORN in view of ENDOH teach the apparatus of claim 16,
BJORN further teaches wherein the apparatus is a wireless communication device (Fig. 1, Page 3, Page 3, Lines [9-11] – BJORN discloses image capture device 112 further comprises a memory and communication module 120 that is arranged to record the captured images and subsequently communicate them wirelessly to a computer 122. Fig. 2, Lines [13-18] – BJORN discloses computer 122 comprises a micro-processor 210 arranged to execute computer readable instructions as may be provided to the computer 122 via one or more of: a network interface 212 arranged to enable the micro-processor 210 to communicate with an external network - for example the internet; a wireless interface 214; a plurality of input interfaces 216 including a keyboard, a mouse, a disk drive and a USB connection;).
Regarding claim 27, BJORN in view of ENDOH teach the apparatus of claim 26,
BJORN further teaches wherein the second image is captured by a camera of the wireless communication device (Fig. 1, Page 3, Lines [9-11] – BJORN discloses image capture device 112 further comprises a memory and communication module 120 that is arranged to record the captured images and subsequently communicate them wirelessly to a computer 122. Lines [23-32] – BJORN further discloses the image capture device 112 is traversed along the tunnel lining 110 whilst images are acquired by the plurality of cameras 114 and stored in the memory and communication module 120. Subsequently, the images recorded on the capture device are transmitted to the computer 122 and stored in the memory 218 thereof. Following such an initial scan of the tunnel lining 110, during a subsequent time period, for example when it is deemed to be time to again inspect the tunnel lining, the image capture device 112 is again positioned within the tunnel lining 110 and one or more further images are required. The further images are transmitted to the computer 122 so that they can be compared with the initially acquired images in order to identify whether any changes to the tunnel lining 110 have occurred.).
Claims 2, 8, 9, 17, and 21-24 are rejected under 35 U.S.C. 103 as being unpatentable over BJORN (GB 2542118 A), hereinafter referenced as BJORN in view of ENDOH (US 20190303766 A1), hereinafter referenced as ENDOH further in view of GONZALEZ (US 20220383128 A1), hereinafter referenced as GONZALEZ.
Regarding claim 2, BJORN in view of ENDOH teach the method of claim 1,
Although ENDOH further teaches wherein each of the set of ML models (Fig. 6, Paragraph [0071] – ENDOH discloses the modules 10A to 10C each include the learning model of the DL and the autoencoder. As another aspect, at the time of recognition, test data is input to the modules 10A to 10C as input data.),
BJORN in view of ENDOH fail to explicitly teach wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure.
However, GONZALES explicitly teaches wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure (Fig. 4, Paragraph [0056] – GONZALES discloses the anomaly localization model 400 includes a convolutional neural network autoencoder that includes an encoder (e.g., corresponding to the convolutional neural network encoder described in connection with a feature extraction model described above) and a decoder (e.g., corresponding to the convolutional neural network decoder described in connection with the anomaly localization model above) that are trained according to examples described elsewhere herein.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; with the teachings of GONZALEZ of having wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure.
Wherein BJORN’s method for detecting changes in a physical environment wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and GONZALEZ relate to machine learning models for implementing anomaly detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and GONZALEZ relates to an object analysis system for detecting, classifying, and/or locating an anomaly on an object that may utilize a robust and accurate image-based anomaly detection model to facilitate management and/or processing of objects and ensure that analyzed objects satisfy certain criteria or standards (e.g., are non-anomalous objects) before the objects are output from the object management system, used in the field, and/or sold to consumers, thereby reducing or preventing a likelihood of a hazard or a degraded consumer experience from objects that do not satisfy the certain criteria or standards. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and GONZALEZ (US 20220383128 A1), Paragraph [0018, 0049].
Regarding claim 8, BJORN in view of ENDOH teach the method of claim 1,
BJORN further discloses wherein the first image and the second image are divided into grid cells (Fig. 4, Page 9, Lines [26-28] – BJORN discloses the network accepts pairs of approximately registered image patches [wherein image patches are grid cells] taken at different times and classifies them to detect anomalous changes.),
wherein the detecting if there are changes in the physical environment is based on grid cells of the second image that are dissimilar from corresponding grid cells of the first image (Page 5, Lines [16-21] – BJORN discloses the CNN outputs a change mask indicative of the presence or absence of a change to the structure between the first and second time periods. As one possibility, the change mask is a binary array the same size as one or both of the images that were used as the first and second channel inputs and indicates, on a pixel-by-pixel basis the presence of a change with a '1' and the absence of a change with a 'O' (or vice versa). See also Page 11, Lines [2-3].).
BJORN fails to explicitly teach and each grid cell is associated with a feature vector, wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold,
However, GONZALEZ explicitly teaches and each grid cell is associated with a feature vector (Fig. 5, Paragraph [0061] – GONZALEZ discloses the machine learning system may identify a feature set (e.g., one or more features and/or feature values) by extracting the feature set from structured data (e.g., image data associated with images that depict non-anomalous objects), by performing an image processing technique to extract the feature set from unstructured data (e.g., image data associated with images that depict anomalous objects and non-anomalous objects), and/or by receiving input from an operator.),
wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold (Fig. 5, Paragraph [0067] – GONZALEZ discloses ) the machine learning system may apply the trained machine learning model 525 to a new observation, such as by receiving a new observation and inputting the new observation to the trained machine learning model 525. As shown, the new observation may include a first feature of Contour_N, a second feature of RGB_N, a third feature of (X.sub.N, Y.sub.N), and so on, as an example. For example, the output may include a predicted value of a target variable, such as when supervised learning is employed. Additionally, or alternatively, the output may include information that identifies a cluster to which the new observation belongs and/or information that indicates a degree of similarity between the new observation and one or more other observations, such as when unsupervised learning is employed. See also Paragraph [0070, 0092].),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; with the teachings of GONZALEZ of having and each grid cell is associated with a feature vector, wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold.
Wherein BJORN’s method for detecting changes in a physical environment wherein and each grid cell is associated with a feature vector, wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and GONZALEZ relate to machine learning models for implementing anomaly detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and GONZALEZ relates to an object analysis system for detecting, classifying, and/or locating an anomaly on an object that may utilize a robust and accurate image-based anomaly detection model to facilitate management and/or processing of objects and ensure that analyzed objects satisfy certain criteria or standards (e.g., are non-anomalous objects) before the objects are output from the object management system, used in the field, and/or sold to consumers, thereby reducing or preventing a likelihood of a hazard or a degraded consumer experience from objects that do not satisfy the certain criteria or standards. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and GONZALEZ (US 20220383128 A1), Paragraph [0018, 0049].
Regarding claim 9, BJORN in view of ENDOH teach the method of claim 1,
BJORN further discloses wherein the first image and the second image are divided into grid cells (Fig. 4, Page 9, Lines [26-28] – BJORN discloses the network accepts pairs of approximately registered image patches [wherein image patches are grid cells] taken at different times and classifies them to detect anomalous changes.),
wherein the detecting if there are changes in the physical environment is based on grid cells of the second image that are dissimilar from the corresponding grid cells of the first image and have a number of neighboring grid cells dissimilar from corresponding grid cells of the first image (Page 5, Lines [16-21] – BJORN discloses the CNN outputs a change mask indicative of the presence or absence of a change to the structure between the first and second time periods. As one possibility, the change mask is a binary array the same size as one or both of the images that were used as the first and second channel inputs and indicates, on a pixel-by-pixel basis the presence of a change with a '1' and the absence of a change with a 'O' (or vice versa). See also Page 11, Lines [2-3].).
BJORN fails to explicitly teach and each grid cell is associated with a feature vector, wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold,
However, GONZALEZ explicitly teaches and each grid cell is associated with a feature vector (Fig. 5, Paragraph [0061] – GONZALEZ discloses the machine learning system may identify a feature set (e.g., one or more features and/or feature values) by extracting the feature set from structured data (e.g., image data associated with images that depict non-anomalous objects), by performing an image processing technique to extract the feature set from unstructured data (e.g., image data associated with images that depict anomalous objects and non-anomalous objects), and/or by receiving input from an operator.),
wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold (Fig. 5, Paragraph [0067] – GONZALEZ discloses ) the machine learning system may apply the trained machine learning model 525 to a new observation, such as by receiving a new observation and inputting the new observation to the trained machine learning model 525. As shown, the new observation may include a first feature of Contour_N, a second feature of RGB_N, a third feature of (X.sub.N, Y.sub.N), and so on, as an example. For example, the output may include a predicted value of a target variable, such as when supervised learning is employed. Additionally, or alternatively, the output may include information that identifies a cluster to which the new observation belongs and/or information that indicates a degree of similarity between the new observation and one or more other observations, such as when unsupervised learning is employed. See also Paragraph [0070, 0092].),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; using the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; with the teachings of GONZALEZ of having and each grid cell is associated with a feature vector, wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold.
Wherein BJORN’s method for detecting changes in a physical environment wherein and each grid cell is associated with a feature vector, wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and GONZALEZ relate to machine learning models for implementing anomaly detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and GONZALEZ relates to an object analysis system for detecting, classifying, and/or locating an anomaly on an object that may utilize a robust and accurate image-based anomaly detection model to facilitate management and/or processing of objects and ensure that analyzed objects satisfy certain criteria or standards (e.g., are non-anomalous objects) before the objects are output from the object management system, used in the field, and/or sold to consumers, thereby reducing or preventing a likelihood of a hazard or a degraded consumer experience from objects that do not satisfy the certain criteria or standards. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and GONZALEZ (US 20220383128 A1), Paragraph [0018, 0049].
Regarding claim 17, BJORN in view of ENDOH teach the apparatus of claim 16,
Although ENDOH further teaches wherein each of the set of ML models (Fig. 6, Paragraph [0071] – ENDOH discloses the modules 10A to 10C each include the learning model of the DL and the autoencoder. As another aspect, at the time of recognition, test data is input to the modules 10A to 10C as input data.),
BJORN in view of ENDOH fail to explicitly teach wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure.
However, GONZALES explicitly teaches wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure (Fig. 4, Paragraph [0056] – GONZALES discloses the anomaly localization model 400 includes a convolutional neural network autoencoder that includes an encoder (e.g., corresponding to the convolutional neural network encoder described in connection with a feature extraction model described above) and a decoder (e.g., corresponding to the convolutional neural network decoder described in connection with the anomaly localization model above) that are trained according to examples described elsewhere herein.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; use the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; with the teachings of GONZALEZ of having wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure.
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein each of the set of ML models is an ML model with a Convolutional Autoencoder, CAE, structure.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and GONZALEZ relate to machine learning models for implementing anomaly detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and GONZALEZ relates to an object analysis system for detecting, classifying, and/or locating an anomaly on an object that may utilize a robust and accurate image-based anomaly detection model to facilitate management and/or processing of objects and ensure that analyzed objects satisfy certain criteria or standards (e.g., are non-anomalous objects) before the objects are output from the object management system, used in the field, and/or sold to consumers, thereby reducing or preventing a likelihood of a hazard or a degraded consumer experience from objects that do not satisfy the certain criteria or standards. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and GONZALEZ (US 20220383128 A1), Paragraph [0018, 0049].
Regarding claim 21, BJORN in view of ENDOH teach the apparatus of claim 16,
Although BJORN explicitly teaches wherein the first image and the second image are divided into grid cells (Fig. 4, Page 9, Lines [26-28] – BJORN discloses the network accepts pairs of approximately registered image patches [wherein image patches are grid cells] taken at different times and classifies them to detect anomalous changes.),
BJORN fails to explicitly teach and each grid cell is associated with a feature vector.
However, GONZALEZ explicitly teaches and each grid cell is associated with a feature vector (Fig. 5, Paragraph [0061] – GONZALEZ discloses the machine learning system may identify a feature set (e.g., one or more features and/or feature values) by extracting the feature set from structured data (e.g., image data associated with images that depict non-anomalous objects), by performing an image processing technique to extract the feature set from unstructured data (e.g., image data associated with images that depict anomalous objects and non-anomalous objects), and/or by receiving input from an operator.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; use the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; with the teachings of GONZALEZ of having wherein and each grid cell is associated with a feature vector.
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein and each grid cell is associated with a feature vector.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and GONZALEZ relate to machine learning models for implementing anomaly detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and GONZALEZ relates to an object analysis system for detecting, classifying, and/or locating an anomaly on an object that may utilize a robust and accurate image-based anomaly detection model to facilitate management and/or processing of objects and ensure that analyzed objects satisfy certain criteria or standards (e.g., are non-anomalous objects) before the objects are output from the object management system, used in the field, and/or sold to consumers, thereby reducing or preventing a likelihood of a hazard or a degraded consumer experience from objects that do not satisfy the certain criteria or standards. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and GONZALEZ (US 20220383128 A1), Paragraph [0018, 0049].
Regarding claim 22, BJORN and ENDOH in view of GONZALEZ teach the apparatus of claim 21,
BJORN in view of ENDOH fail to explicitly teach wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold.
However, GONZALEZ explicitly teaches wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold (Fig. 5, Paragraph [0067] – GONZALEZ discloses ) the machine learning system may apply the trained machine learning model 525 to a new observation, such as by receiving a new observation and inputting the new observation to the trained machine learning model 525. As shown, the new observation may include a first feature of Contour_N, a second feature of RGB_N, a third feature of (X.sub.N, Y.sub.N), and so on, as an example. For example, the output may include a predicted value of a target variable, such as when supervised learning is employed. Additionally, or alternatively, the output may include information that identifies a cluster to which the new observation belongs and/or information that indicates a degree of similarity between the new observation and one or more other observations, such as when unsupervised learning is employed. See also Paragraph [0070, 0092].).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; use the second image as input to a set of machine learning, ML, models to generate a reconstructed image of the second image from each of the set of ML models; with the teachings of GONZALEZ of having wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold.
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein a grid cell of the second image is dissimilar from a corresponding grid cell of the first image if a distance between a feature vector associated with the grid cell of the second image and a feature vector associated with the corresponding grid cell of the first image is above a dissimilarity threshold.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and GONZALEZ relate to machine learning models for implementing anomaly detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and GONZALEZ relates to an object analysis system for detecting, classifying, and/or locating an anomaly on an object that may utilize a robust and accurate image-based anomaly detection model to facilitate management and/or processing of objects and ensure that analyzed objects satisfy certain criteria or standards (e.g., are non-anomalous objects) before the objects are output from the object management system, used in the field, and/or sold to consumers, thereby reducing or preventing a likelihood of a hazard or a degraded consumer experience from objects that do not satisfy the certain criteria or standards. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and GONZALEZ (US 20220383128 A1), Paragraph [0018, 0049].
Regarding claim 23, BJORN and ENDOH in view of GONZALEZ teach the apparatus of claim 22,
BJORN further teaches wherein to detect if there are changes in the physical environment is based on grid cells of the second image that are dissimilar from the corresponding grid cells of the first image (Page 5, Lines [16-21] – BJORN discloses the CNN outputs a change mask indicative of the presence or absence of a change to the structure between the first and second time periods. As one possibility, the change mask is a binary array the same size as one or both of the images that were used as the first and second channel inputs and indicates, on a pixel-by-pixel basis the presence of a change with a '1' and the absence of a change with a 'O' (or vice versa). See also Page 11, Lines [2-3].).
Regarding claim 24, BJORN and ENDOH in view of GONZALEZ teach the apparatus of claim 22,
BJORN further teaches wherein to detect if there are changes in the physical environment is based on grid cells of the second image that are dissimilar from the corresponding grid cells of the first image and have a number of neighboring grid cells dissimilar from the corresponding grid cells of the first image (Page 5, Lines [16-21] – BJORN discloses the CNN outputs a change mask indicative of the presence or absence of a change to the structure between the first and second time periods. As one possibility, the change mask is a binary array the same size as one or both of the images that were used as the first and second channel inputs and indicates, on a pixel-by-pixel basis the presence of a change with a '1' and the absence of a change with a 'O' (or vice versa). See also Page 11, Lines [2-3].).
Claims 5, 10, 20, and 25 are rejected under 35 U.S.C. 103 as being unpatentable over BJORN (GB 2542118 A), hereinafter referenced as BJORN in view of ENDOH (US 20190303766 A1), hereinafter referenced as ENDOH further in view of WANG (US 20220230291 A1), hereinafter referenced as WANG.
Regarding claim 5, BJORN in view of ENDOH teach the method of claim 1,
Although BJORN further teaches Normalized Cross-Correlation (NCC) (Figs. 9-10, Page 16, Lines [5-7] – BJORN discloses RGB shows the performance of pixel-to-pixel absolute differencing; and the known method is applied using NCC windows of varying sizes from 5x5 to 15x 15 pixels.),
BJORN fails to explicitly teach wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE),
However, ENDOH explicitly teaches wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE) (Fig. 15, Paragraph [0120] – ENDOH discloses the reconstruction error is calculated by calculating the square of a difference between pixel values for each pair of pixels the positions of which correspond to each other between two pieces of image data, and performing predetermined statistical processing, for example, averaging processing, on the square of the difference between pixel values calculated for each pair of pixels.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image, with the teachings of ENDOH of having wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE).
Wherein BJORN’s method for detecting changes in a physical environment wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE).
The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and ENDOH relate to machine learning models for implementing recognition using input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and ENDOH relates to an adaptability calculation method, an adaptability calculation device, a computer-readable recording medium, an identification method, and an identification device with respect to input data, wherein an output from a wrong learning model is prevented from being selected, so that a possibility of failing in recognition can be reduced. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and ENDOH (US 20190303766 A1), Paragraph [0002, 0067].
BJORN in view of ENDOH fail to explicitly teach Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
However, WANG explicitly teaches Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR) (Fig. 4, Paragraph [0036] – WANG discloses the type of the testing errors can be peak signal to Noise Ratio (PSNR), or structural similarity (SSIM), not being limited.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; selecting an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image, with the teachings of WANG of having Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
Wherein BJORN’s method for detecting changes in a physical environment wherein the smallest reconstruction error is calculated based on at least one of: Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and WANG relate to machine learning models for implementing image detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and WANG relates to a method for detecting product defects in images of the products, wherein accuracy of the comparison between the candidate be-analyzed reconstructed image and the candidate be-analyzed target image is improved, so increasing detection accuracy. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and WANG (US 20220230291 A1), Paragraph [0013, 0053].
Regarding claim 10, BJORN in view of ENDOH teach the method of claim 1,
BJORN in view of ENDOH fail to explicitly teach further comprising: in response to detecting that there are changes in the physical environment, initiating a message to a user indicating that the physical environment has changed.
However, WANG explicitly teaches further comprising: in response to detecting that there are changes in the physical environment, initiating a message to a user indicating that the physical environment has changed (Fig. 5, Paragraph [0061] – WANG discloses the defect detection apparatus 100 can further include a prompting module 106. The prompting module 106 outputs a warning or a prompt according to the result. For example, in one embodiment, when the result is taken as confirming that there is one or more defect exist and are revealed in the to-be-analyzed image, the prompting module 106 outputs the prompt, and is sent to a terminal device of a specified contact person. The specified person can be a quality person in charge of detecting defects in the images of target objects. Thus, when the image with the defects, the specified person is notified.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model, with the teachings of WANG of having further comprising: in response to detecting that there are changes in the physical environment, initiating a message to a user indicating that the physical environment has changed.
Wherein BJORN’s method for detecting changes in a physical environment wherein further comprising: in response to detecting that there are changes in the physical environment, initiating a message to a user indicating that the physical environment has changed.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and WANG relate to machine learning models for implementing image detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and WANG relates to a method for detecting product defects in images of the products, wherein accuracy of the comparison between the candidate be-analyzed reconstructed image and the candidate be-analyzed target image is improved, so increasing detection accuracy. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and WANG (US 20220230291 A1), Paragraph [0013, 0053].
Regarding claim 20, BJORN in view of ENDOH teach the apparatus of claim 16,
Although BJORN further teaches Normalized Cross-Correlation (NCC) (Figs. 9-10, Page 16, Lines [5-7] – BJORN discloses RGB shows the performance of pixel-to-pixel absolute differencing; and the known method is applied using NCC windows of varying sizes from 5x5 to 15x 15 pixels.),
BJORN fails to explicitly teach wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE),
However, ENDOH explicitly teaches wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE) (Fig. 15, Paragraph [0120] – ENDOH discloses the reconstruction error is calculated by calculating the square of a difference between pixel values for each pair of pixels the positions of which correspond to each other between two pieces of image data, and performing predetermined statistical processing, for example, averaging processing, on the square of the difference between pixel values calculated for each pair of pixels.),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; select an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image; with the teachings of ENDOH of having wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE).
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein the smallest reconstruction error is calculated based on at least one of: Mean Squared Error (MSE).
The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and ENDOH relate to machine learning models for implementing recognition using input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and ENDOH relates to an adaptability calculation method, an adaptability calculation device, a computer-readable recording medium, an identification method, and an identification device with respect to input data, wherein an output from a wrong learning model is prevented from being selected, so that a possibility of failing in recognition can be reduced. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and ENDOH (US 20190303766 A1), Paragraph [0002, 0067].
BJORN in view of ENDOH fail to explicitly teach Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
However, WANG explicitly teaches Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR) (Fig. 4, Paragraph [0036] – WANG discloses the type of the testing errors can be peak signal to Noise Ratio (PSNR), or structural similarity (SSIM), not being limited.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; select an ML model among the set of ML models with a smallest reconstruction error between the second image and the reconstructed image of the second image; with the teachings of WANG of having Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein the smallest reconstruction error is calculated based on at least one of: Structural Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and WANG relate to machine learning models for implementing image detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and WANG relates to a method for detecting product defects in images of the products, wherein accuracy of the comparison between the candidate be-analyzed reconstructed image and the candidate be-analyzed target image is improved, so increasing detection accuracy. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and WANG (US 20220230291 A1), Paragraph [0013, 0053].
Regarding claim 25, BJORN in view of ENDOH teach the apparatus of claim 16,
BJORN in view of ENDOH fail to explicitly teach the processing circuitry is further configured to cause the apparatus to: in response to detecting that there are changes in the physical environment, initiate a message to a user indicating that the physical environment has changed.
However, WANG explicitly teaches the processing circuitry is further configured to cause the apparatus to (Fig. 6, Paragraph [0065] – WANG discloses the processor 202 is configured to execute the computer programs 203 to implement the blocks in the method, for example the block S1 to the block S7. The processor 202 is configured to execute the computer programs 203 to implement the function of the modules in the defect detection apparatus 100, for example, the training module 101, the image processing module 102, the comparing module 103, the confirming module 104, the obtaining module 105, and the prompting module 106.):
in response to detecting that there are changes in the physical environment, initiate a message to a user indicating that the physical environment has changed (Fig. 5, Paragraph [0061] – WANG discloses the defect detection apparatus 100 can further include a prompting module 106. The prompting module 106 outputs a warning or a prompt according to the result. For example, in one embodiment, when the result is taken as confirming that there is one or more defect exist and are revealed in the to-be-analyzed image, the prompting module 106 outputs the prompt, and is sent to a terminal device of a specified contact person. The specified person can be a quality person in charge of detecting defects in the images of target objects. Thus, when the image with the defects, the specified person is notified.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; and detect if there are changes in the physical environment by using the first image and the second image as input to the selected ML model, with the teachings of WANG of having the processing circuitry is further configured to cause the apparatus to: in response to detecting that there are changes in the physical environment, initiate a message to a user indicating that the physical environment has changed.
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein the processing circuitry is further configured to cause the apparatus to: in response to detecting that there are changes in the physical environment, initiate a message to a user indicating that the physical environment has changed.
The motivation behind this modification would have been to provide a method for anomaly detection with improved accuracy, since both BJORN and WANG relate to machine learning models for implementing image detection from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and WANG relates to a method for detecting product defects in images of the products, wherein accuracy of the comparison between the candidate be-analyzed reconstructed image and the candidate be-analyzed target image is improved, so increasing detection accuracy. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and WANG (US 20220230291 A1), Paragraph [0013, 0053].
Claims 4 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over BJORN (GB 2542118 A), hereinafter referenced as BJORN in view of ENDOH (US 20190303766 A1), hereinafter referenced as ENDOH further in view of KOSSYK (US 20240338565 A1), hereinafter referenced as KOSSYK.
Regarding claim 4, BJORN in view of ENDOH teach the method of claim 1,
Although BJORN further teaches wherein the detecting if there are changes in the physical environment (Fig. 3, Page 5, Lines [10-12], See also Page 7, Lines [21-28].)
BJORN in view of ENDOH fail to explicitly teach wherein the detecting if there are changes in the physical environment comprises comparing feature vectors of the first image with feature vectors of the second image, wherein the feature vectors are generated by the selected ML model.
However, KOSSYK explicitly teaches wherein the detecting if there are changes in the physical environment (Fig. 2, Paragraph [0032] – KOSSYK discloses FIG. 2 illustrates a process for determining a change in a feature at a specified time)
comprises comparing feature vectors of the first image with feature vectors of the second image (Fig. 2, Paragraph [0035] – KOSSYK discloses at step 206, the server generates one or more time series feature vectors based on the feature, each of which corresponds to the feature from the each of the plurality of time series images extracted at step 204. Paragraph [0036] – KOSSYK further discloses at step 208, the server creates a neural network model configured to predict a change in the feature based on the plurality of time series feature vectors generated in step 206. Paragraph [0037] – KOSSYK further discloses At step 210, the server determines, using the neural network model, a change in the feature at a specified time based on a change between/among the time series feature vectors.),
wherein the feature vectors are generated by the selected ML model (Fig. 1, Paragraph [0030] – KOSSYK discloses for each of the time-series images 101_1, 101_2, . . . , 101_n, the CNN 110 can extract a feature vector x, resulting in a time series of feature vectors x.sub.0 121_1, x.sub.1 121_2, . . . , x.sub.n 121_n, which respectively correspond to the time-series images 101_1, 101_2, . . . , 101_n.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having a method for detecting changes in a physical environment, the method performed by an apparatus and comprising: obtaining a first image representing the physical environment at a first time instance; obtaining a second image representing the physical environment at a second time instance; and detecting if there are changes in the physical environment by using the first image and the second image as input to the selected ML model, with the teachings of KOSSYK of having wherein the detecting if there are changes in the physical environment comprises comparing feature vectors of the first image with feature vectors of the second image, wherein the feature vectors are generated by the selected ML model.
Wherein BJORN’s method for detecting changes in a physical environment wherein the detecting if there are changes in the physical environment comprises comparing feature vectors of the first image with feature vectors of the second image, wherein the feature vectors are generated by the selected ML model.
The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and KOSSYK relate to machine learning models for detecting changes from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and KOSSYK relates to systems, methods, and computer readable media can be used to detect changes in time series data, provide predictive analytics based on the detected changes, and perform predictive maintenance using time series data. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and KOSSYK (US 20240338565 A1), Paragraph [0019].
Regarding claim 19, BJORN in view of ENDOH teach the apparatus of claim 16,
Although BJORN further teaches wherein to detect if there are changes in the physical environment (Fig. 3, Page 5, Lines [10-12], See also Page 7, Lines [21-28].)
BJORN in view of ENDOH fail to explicitly teach wherein to detect if there are changes in the physical environment comprises to compare feature vectors of the first image with feature vectors of the second image, wherein the feature vectors are generated by the selected ML model.
However, KOSSYK explicitly teaches wherein to detect if there are changes in the physical environment (Fig. 2, Paragraph [0032] – KOSSYK discloses FIG. 2 illustrates a process for determining a change in a feature at a specified time)
comprises to compare feature vectors of the first image with feature vectors of the second image (Fig. 2, Paragraph [0035] – KOSSYK discloses at step 206, the server generates one or more time series feature vectors based on the feature, each of which corresponds to the feature from the each of the plurality of time series images extracted at step 204. Paragraph [0036] – KOSSYK further discloses at step 208, the server creates a neural network model configured to predict a change in the feature based on the plurality of time series feature vectors generated in step 206. Paragraph [0037] – KOSSYK further discloses At step 210, the server determines, using the neural network model, a change in the feature at a specified time based on a change between/among the time series feature vectors.),
wherein the feature vectors are generated by the selected ML model (Fig. 1, Paragraph [0030] – KOSSYK discloses for each of the time-series images 101_1, 101_2, . . . , 101_n, the CNN 110 can extract a feature vector x, resulting in a time series of feature vectors x.sub.0 121_1, x.sub.1 121_2, . . . , x.sub.n 121_n, which respectively correspond to the time-series images 101_1, 101_2, . . . , 101_n.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date the claimed invention was made to combine the teachings of BJORN in view of ENDOH of having an apparatus for detecting changes in a physical environment, the apparatus comprising a processing circuitry causing the apparatus to be operative to: obtain a first image representing the physical environment at a first time instance; obtain a second image representing the physical environment at a second time instance; and detect if there are changes in the physical environment by using the first image and the second image as input to the selected ML model, with the teachings of KOSSYK of having wherein to detect if there are changes in the physical environment comprises to compare feature vectors of the first image with feature vectors of the second image, wherein the feature vectors are generated by the selected ML model.
Wherein BJORN’s apparatus for detecting changes in a physical environment wherein to detect if there are changes in the physical environment comprises to compare feature vectors of the first image with feature vectors of the second image, wherein the feature vectors are generated by the selected ML model.
The motivation behind this modification would have been to provide an enhanced method for reliably and accurately detecting changes from input data, since both BJORN and KOSSYK relate to machine learning models for detecting changes from input data, wherein BJORN relates to the detection of temporal changes in a structure using a neural network that has been trained to identify changes in images, wherein the approach can be straightforwardly adapted to different textured surfaces and new scenarios with minimal manual training effort; and KOSSYK relates to systems, methods, and computer readable media can be used to detect changes in time series data, provide predictive analytics based on the detected changes, and perform predictive maintenance using time series data. Please see BJORN (GB 2542118 A), Page 1, Lines [4-5], Page 2, Lines [21-22], Page 17, Lines [23-25] Paragraph [0002, 0047], and KOSSYK (US 20240338565 A1), Paragraph [0019].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEZAWIT N SHIMELES whose telephone number is (571)272-7663. The examiner can normally be reached M-F 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEZAWIT NOLAWI SHIMELES/Examiner, Art Unit 2673
Prosecution Insights