DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “an estimator configured to estimate” and “a diagnosis unit configured to diagnose” as recited in claim 1.
Because the claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Examiner’s Note
While not rising to the level of a 112(b) rejection, the examiner notes a potential unintended consequence within the dependency of claims 6-7. To resolve the claims of antecedent basis issues, it is necessary to interpret “the traffic object that is unnecessary for the estimation of the installation state parameter” as being the same traffic object of claim 2 that “can be used for the estimation”. Such that for all given traffic objects, they are merely optional sources of information for estimation under the open-ended transitional phrase “comprising”. This would have the effect of making even traffic objects which are used by the device for estimation, “traffic objects that [are] unnecessary for the estimation”, such that a device which considers all traffic objects would appropriately read upon the claims.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 4-5, 8 and 10-11 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
With respect to claim 8, an antecedent basis issue is present as claim 2 only refers to a traffic object that “can be used” for estimation. Claim 8 refers back to this singular traffic object as the object which “is used” for estimation, when there was no previous indication that it was used at all. The traffic object which is “unnecessary for the estimation” corresponds 1:1 with the object which “can be used” for estimation (ie., an object which is unnecessary for the same) of claim 2, but the object which “is used” is a narrower class of object (by virtue of its actual employed use) and needs an appropriate distinguishing title.
Claims 4-5 and 10-11 are rejected on the same grounds, with their reference to “the traffic object used for the estimation”.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-15 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims of Patent No. US 12322137 B2 (Hereinafter, “137”). Although the claims at issue are not identical, they are not patentably distinct from each other. Reference may be made to the limitation mappings/table presented below in view of representative claims of 137.
The instant claims and the claims of reference recite common subject matter, and recite the open-ended transitional phrase “comprising” which does not preclude any additional elements recited by claims of reference.
Language/terminology of the instant claims constituting minor/slight variations from the
claims of reference, if/where present, require interpretations under Broadest Reasonable
Interpretation and/or plain meaning definitions (MPEP 2173 and 2111) equivalent to/met by
language of the reference claims in view of that corresponding Specification. While the
disclosure of reference may not be used as prior art (Double Patenting concerns the claims of
reference), portions of the specification which provide support for reference claims may also be
examined and considered when addressing the scope of claims of reference and the issue of
whether an instant claim defines an obvious variation or falls within the scope of an invention
claimed in the claim(s) of reference. See MPEP 804 with reference to In re Vogel, 422 F.2d 438,
164 USPQ 619 (CCPA 1970).
Whereby elements of the instant claims otherwise not present explicitly in corresponding reference claims correspond to disclosure as identified in Aguirre et. al (US 20190051015 A1) (Hereinafter, “Aguirre”) and Siam et. al MODNet: Motion and Appearance based Moving Object Detection Network for Autonomous Driving (Hereinafter, “Siam”), it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify claims of reference such that the teachings may further include machine-learning implemented methods for object detection. Such methods, while not explicitly within the claims, can be found throughout the specification of 137 (Col. 6, “The conversion equation or the conversion table is used in conversion of coordinates of an object on a road surface or a floor surface in a captured image or, more specifically, conversion of the coordinates from the image coordinate system to the world coordinate system”; Col. 18, “The sensors 116 held by the holder 117 detect motion of waves propagating from objects that are placed to determine correlation by calculation (i.e., objects for calculation of correlation). The controller 122 calculates the relative positions and the relative orientations of the other sensors 116 with respect to the first sensor 118 on the basis of the detected motion of waves propagating from the objects for calculation of correlation and incident on the sensors 116. This will be described below in detail”; Col. 22, “For example, the bounds of the road surface or the floor surface may be determined by a well-known method, such as machine learning, pattern matching, or free space detection”). The systems readily integrate, as Aguirre/Siam represent a mere elaboration of concepts otherwise reflected in the claims and present within the specification of 137. A person of ordinary skill in the art would be motivated to combine these methods to provide a system which takes further advantage of machine-learning techniques (which as admitted by 137, are “well-known”) to enhance the estimation process. The implementation of a non-transitory memory as relevant to claim 15 is further obvious, for the efficiency of execution by a conventional processor.
Whereby elements of the instant claims otherwise not present explicitly in corresponding reference claims correspond to disclosure as identified in Aguirre/Siam and Beltransen beltransen/lidar_bev (Hereinafter, “Beltransen”), it would have been further obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify claims of reference such that the teachings may further include bird’s-eye view states. Doing so would provide greater information and integration of outputs through reference. The systems readily integrate, as the specification of 137 also expects the usage of LiDAR in a potentially similar manner (Col. 16, “The sensors 116 includes at least one of a…LIDAR instrument”).
Claims of Reference (137)
Instant claims (18/833,241)
Claim 1An image processing device, comprising:
an image interface that acquires a captured image from an imaging module;
a memory in which locations of feature points in a world coordinate system are stored, the feature points being located on a road surface or a floor surface in a first captured image provided by the imaging module, wherein first reference locations of the feature points in a first captured image provided by the imaging module at a specific timing are further stored in the memory; and
a controller that detects second reference locations of the feature points in the second captured image and calculates based on the feature points and the locations in the world coordinate system, a calibration parameter for calibrating a location of the imaging module and an orientation of the imaging module, wherein
the controller detects the second reference locations of the feature points in the second captured image provided by the imaging module at a timing different from the specific timing, and
recalculates the calibration parameter in response to finding a discrepancy between the second reference locations and the first reference locations with regard to a predetermined percentage or more of the feature points
wherein machine learning corresponds to cited portions of Aguirre/Siam below
Claim 1
A state estimation device comprising :an estimator configured to estimate an installation state parameter of an imaging device having obtained image data through imaging, by using a state estimation model subjected to machine learning to estimate the installation state parameter of the imaging device having obtained the input image data through the imaging, the machine learning using first training data comprising the image data obtained by the imaging device having imaged a traffic environment and first correct value data of the installation state parameter of the imaging device having obtained the input image data through the imaging; and a diagnosis unit configured to diagnose an installation state of the imaging device based on the estimated installation state parameter.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 2
The state estimation device according to claim 1, further comprising a processor configured to perform processing such that the image data obtained by the imaging device having imaged the traffic environment comprises a traffic object that can be used for the estimation, wherein the estimator inputs the image data processed by the processor to the state estimation model and estimates the installation state parameter of the imaging device.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 3
The state estimation device according to claim 2, wherein the processor performs processing such that the image data comprises an image in which the traffic object exists in a predetermined area and/or an image in which the traffic object faces toward the imaging device.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 4
The state estimation device according to claim 2, wherein the processor performs processing, based on an estimation result, such that the image data obtained by the imaging device having imaged the traffic environment comprises the traffic object used for the estimation, by using an object estimation model subjected to machine learning to estimate at least one selected from a group consisting of a position, a size, and a type of the traffic object in the traffic environment indicated by the input image data, the second training data comprising the image data and second correct value data for object detection in the image data, and the estimation result being obtained by estimating the at least one selected from the group of the position, the size, and type of the traffic object in the traffic environment indicated by the input image data
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 5
The state estimation device according to claim 4, wherein the object estimation model is a model subjected to machine learning to further estimate a direction of the traffic object in the traffic environment indicated by the input image data, andthe processor performs processing, by using the object estimation model, such that the image data includes the traffic object approaching the imaging device.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 6
The state estimation device according to claim 2, wherein the processor processes the image data such that the traffic object that is unnecessary for the estimation of the installation state parameter of the imaging device is changed or deleted from the image.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 7
The state estimation device according to claim 2, wherein the traffic object that is unnecessary for the estimation of the installation state parameter of the imaging device is at least one selected from a group consisting of a truck, a passenger car, and a construction machine that exists in a predetermined area of an image indicated by the image data.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 8
The state estimation device according to claim 2, wherein the processor selects, from a plurality of pieces of the image data obtained in chronological order through the imaging, the image data that comprises the traffic object that is used for the estimation of the installation state parameter of the imaging device and does not comprise the traffic object that is unnecessary for the estimation.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 9
The state estimation device according to claim 2, wherein the processor adds the traffic object that can be used for the estimation of the installation state parameter to a preset setting area of the image data.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 10
The state estimation device according to claim 2, wherein the processor determines a moving direction of the traffic object based on a plurality of pieces of the image data obtained in chronological order through the imaging, and performs the processing based on a determination result such that the image data comprises the traffic object used for the estimation.
Claim 1
in further view of cited portions of Aguirre/Siam below
Claim 11
The state estimation device according to claim 10, wherein the processor determines the moving direction of the traffic object using tracking processing, and performs the processing based on the determination result such that the image data comprises the traffic object used for the estimation.
Claim 1
in further view of cited portions of Aguirre/Siam/Beltransen below
Claim 12
The state estimation device according to claim 2, wherein the diagnosis unit diagnoses an installation state of the imaging device based on the installation state parameter estimated by the estimator and a bird's-eye view state of the traffic object indicated by the image data.
Claim 1
in further view of cited portions of Aguirre/Siam/Beltransen below
Claim 13
The state estimation device according to claim 12, wherein the diagnosis unit compares an orientation of the traffic object indicated by the image data, and an orientation of the traffic object calculated based on the installation state parameter estimated by the estimator, and diagnoses the installation state of the imaging device when a degree of coincidence is higher than a determination threshold value.
Claim 1An image processing device, comprising:
an image interface that acquires a captured image from an imaging module;
a memory in which locations of feature points in a world coordinate system are stored, the feature points being located on a road surface or a floor surface in a first captured image provided by the imaging module, wherein first reference locations of the feature points in a first captured image provided by the imaging module at a specific timing are further stored in the memory; and
a controller that detects second reference locations of the feature points in the second captured image and calculates based on the feature points and the locations in the world coordinate system, a calibration parameter for calibrating a location of the imaging module and an orientation of the imaging module, wherein
the controller detects the second reference locations of the feature points in the second captured image provided by the imaging module at a timing different from the specific timing, and
recalculates the calibration parameter in response to finding a discrepancy between the second reference locations and the first reference locations with regard to a predetermined percentage or more of the feature points
wherein machine learning corresponds to cited portions of Aguirre/Siam below
Claim 14
A state estimation method comprising: estimating, by a computer, an installation state parameter of an imaging device having obtained image data through imaging, by using a state estimation model subjected to machine learning to estimate the installation state parameter of the imaging device having obtained the input image data through the imaging, the machine learning using first training data comprising the image data obtained by the imaging device having imaged a traffic environment and first correct value data of the installation state parameter of the imaging device having obtained the input image data through the imaging; and diagnosing, by the computer, an installation state of the imaging device based on the estimated installation state parameter.
Claim 1
An image processing device, comprising:
an image interface that acquires a captured image from an imaging module;
a memory in which locations of feature points in a world coordinate system are stored, the feature points being located on a road surface or a floor surface in a first captured image provided by the imaging module, wherein first reference locations of the feature points in a first captured image provided by the imaging module at a specific timing are further stored in the memory; and
a controller that detects second reference locations of the feature points in the second captured image and calculates based on the feature points and the locations in the world coordinate system, a calibration parameter for calibrating a location of the imaging module and an orientation of the imaging module, wherein
the controller detects the second reference locations of the feature points in the second captured image provided by the imaging module at a timing different from the specific timing, and
recalculates the calibration parameter in response to finding a discrepancy between the second reference locations and the first reference locations with regard to a predetermined percentage or more of the feature points
wherein machine learning and non-transitory computer readable recording medium correspond to cited portions of Aguirre/Siam below
Claim 15
Anon-transitory computer readable recording medium storing therein a state estimation program causing a computer to execute: estimating an installation state parameter of an imaging device having obtained image data through imaging, by using a state estimation model subjected to machine learning to estimate the installation state parameter of the imaging device having obtained the input image data through the imaging, the machine learning using first training data comprising the image data obtained by the imaging device having imaged a traffic environment and first correct value data of the installation state parameter of the imaging device having obtained the input image data through the imaging; and diagnosing an installation state of the imaging device based on the estimated installation state parameter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-11 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Aguirre et. al (US 20190051015 A1) (Hereinafter, “Aguirre”) in view of Siam et. al MODNet: Motion and Appearance based Moving Object Detection Network for Autonomous Driving (Hereinafter, “Siam”).
With respect to claim 1, Aguirre teaches:
A state estimation device ([Abstract]) comprising:
an estimator configured to estimate an installation state parameter of an imaging device having obtained image data through imaging by using a state estimation model subjected to machine learning to estimate the installation state parameter ([0003] “These deformations of chassis or other types of significant changes in the sensor state (e.g., position, orientation, shape, sensibility, etc.) or other forms of damage in the mechanical coupling (such as robotic actuators) would render the autonomous vehicle disabled despite of the possibility to remain in a limited but reliable functional state within the so-called safety-completeness features”; read in line with paragraph [0015] of the claimed invention’s specification) of the imaging device having obtained the input image data through the imaging (Fig. 3A; Fig. 3B; [0028]-[0029] “time-speed estimation” block 340 mapping to “estimator”)
the machine learning (Fig. 3B; [0029] “For that goal the use of featureless labeled segmentation also known as semantic segmentation or signal classification (i.e., Recurrent Deep Neural Networks) may be used to determine similarity or deviating sensor pairs”) using first Fig. 3A; Fig. 3B; [0029] “In the event that the self-calibration completes successfully and all sensors appear to be in a normal state (operation 312) it results in an optimal online calibration 350 for the sensors and control passes back to operation 310. By contrast, if one or more of the sensors appears not to be in a normal state then control passes to operation 314 and a pair-wise analysis is performed between sensors on the vehicle to determine whether there is an edge co-occurrence in the data collected by the sensors. In one example the time registered raw signals from a first sensor and a second sensor are isochronously input to a pair wise co-occurrence association module 346, tracks the latest sensor readings which have a high index of reliability, and computes the consistency of pair-wise (edge of co-occurrence) associatively based on geometric overlapping, to address this a precise timing (isochronous triggering signals) need to be present in the sensors”)
a diagnosis unit configured to diagnose an installation state of the imaging device based on the estimated installation state parameter. (Fig. 3A, block 320; Fig. 3B)
Aguirre does not explicitly teach:
training data
However, Siam, in the same field of endeavor of machine-learning implemented object detection in traffic, teaches:
training data ([4A] “Then the moving object detection network (MODNet) is trained and evaluated on KITTI MOD dataset”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Aguirre to include the limitations of training data. Aguirre is already configured to utilize machine-learning and neural network techniques, both of which would predictably benefit through the use of a training process and requisite training data. The systems readily integrate for the same reason.
With respect to claim 2, Aguirre and Siam teach:
The state estimation device according to claim 1, further comprising a processor configured to perform processing (Aguirre, Fig. 1; Aguirre, [0020]-[0023]) such that the image data obtained by the imaging device having imaged the traffic environment comprises a traffic object that can be used for the estimation (Aguirre, [0029] “For that goal the use of featureless labeled segmentation also known as semantic segmentation or signal classification (i.e., Recurrent Deep Neural Networks) may be used to determine similarity or deviating sensor pairs”; noting that in the context of an autonomous vehicle, these semantically segmented objects are expected to include the broad class of “traffic objects” listed in [0012] of the claimed invention’s specification; Siam, Fig. 3)
wherein the estimator inputs the image data processed by the processor to the state estimation model and estimates the installation state parameter of the imaging device (Aguirre, Fig. 3A; Aguirre, Fig. 3B)
With respect to claim 3, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the processor performs processing such that the image data comprises an image in which the traffic object exists in a predetermined area (Aguirre, [0029] wherein “predetermined area” corresponds to the set viewing range of the sensors determined by their parameters, such as “focal length”)
With respect to claim 4, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the processor performs processing, based on an estimation result, such that the image data obtained by the imaging device having imaged the traffic environment comprises the traffic object used for the estimation, by using an object estimation model subjected to machine learning to estimate at least one selected from a group Siam, Fig. 3; Aguirre, [0029] noting that semantic segmentation distinguishes types of objects), the second training data comprising the image data and second correct value data (Aguirre Fig. 3A; Aguirre, Fig. 3B; noting “first” and “second” correct value data is met as the device operates on a continuous stream of data values in real time across multiple sensors, including data deemed correct under the flow) for object detection in the image data, and the estimation result being obtained by estimating the at least one selected from the group of the Aguirre, [0029])
With respect to claim 5, Aguirre and Siam teach:
The state estimation device according to claim 4, wherein the object estimation model is a model subjected to machine learning to further estimate a direction of the traffic object in the traffic environment indicated by the input image data (Aguirre [0028]-[0030] not explicitly describing motion direction of traffic objects but generally understood to involve an optical-flow type analysis for detections; Siam, [2] “The computed velocity vector per bounding box is compared to the odometry ground-truth to determine the static/moving classification of vehicles”; Siam, Fig. 2, indicating direction of motion)
the processor performs processing, by using the object estimation model, such that the image data includes the traffic object approaching the imaging device (Aguirre [0028]-[0030]; Siam, Fig. 2)
With respect to claim 6, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the processor processes the image data such that the traffic object that is unnecessary for the estimation of the installation state parameter of the imaging device is changed or deleted from the image (Aguirre, under claim interpretation discussed in section above; Siam, [Abstract] “For autonomous driving, moving objects like vehicles and pedestrians are of critical importance as they primarily influence the maneuvering and braking of the car”; Siam, [2] “The objects that are then consistently identified on multiple frames as moving are kept. In this dataset, the focus is on vehicles with car, truck and van object categories; Siam, Fig. 2 “Blue boxes for moving vehicles, green boxes for static ones”)
With respect to claim 7, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the traffic object that is unnecessary for the estimation of the installation state parameter is at least one selected from a group consisting of a truck, a passenger car, and a construction machine that exists in a predetermined area of an image indicated by the image (Siam, Fig. 2; Aguirre, under claim interpretation note discussed above)
With respect to claim 8, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the processor selects, from a plurality of pieces of the image data obtained in chronological order through the imaging, the image data that comprises the traffic object that is used for the estimation of the installation state parameter of the imaging device and does not comprise the traffic object that is unnecessary for the estimation (Siam, Fig. 2, green boxes corresponding to moving objects; Siam, [2] “The objects that are then consistently identified on multiple frames as moving are kept”)
With respect to claim 9, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the processor adds the traffic object that can be used for the estimation of the installation state parameter to a preset setting area of the image data (Siam, [2]; Siam Fig. 2; Aguirre, noting that no special definition is given to “a preset setting area” of the image data. Given that semantic segmentation results in determined masks overlaying the image data, the examiner determines that [0029] reads, as the area included within the mask).
With respect to claim 10, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the processor determines a moving direction of the traffic object based on a plurality of pieces of the image data obtained in chronological order through the imaging, and performs the processing based on a determination result such that the image data comprises the traffic object used for the estimation (Siam, Fig. 2; Siam, [2] “The objects that are then consistently identified on multiple frames as moving are kept”)
With respect to claim 11, Aguirre and Siam teach:
The state estimation device according to claim 10, wherein the processor determines the moving direction of the traffic object using tracking processing, and performs the processing based on the determination result such that the image data comprises the traffic object used for the estimation (Siam, Fig. 2; Siam, [2])
With respect to claim 14, it is functionally parallel to claim 1, but recited as a method performed by a computer. Aguirre/Siam teaches the same (Aguirre, Fig. 6; Aguirre, [0049]). Accordingly, the claim is rejected in line with the analysis above.
With respect to claim 15, it is functionally parallel to claim 1, but recited as a non-transitory computer-readable recording medium storing a program. Aguirre/Siam teaches the same (Aguirre, Fig. 6; Aguirre, [0054]). Accordingly, the claim is rejected in line with the analysis above.
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Aguirre and Siam in view of Beltransen beltransen/lidar_bev (Hereinafter, “Beltransen”)
With respect to claim 12, Aguirre and Siam teach:
The state estimation device according to claim 2, wherein the diagnosis unit diagnoses an installation state of the imaging device based on the installation state parameter estimated by the estimator and a
Aguirre and Siam do not explicitly teach:
a bird’s eye view state
However, Beltransen, in the same field of endeavor of imaging sensors, teaches:
a bird’s eye view state (Beltransen, “scripts”)
It would have been obvious to one of ordinary skill in the art as of the effective filing date of the claimed invention, to modify Aguirre and Siam to include the limitations of a bird’s-eye view, as taught by Beltransen. Doing so would have the advantage of providing more information to the system and a means to correlate multiple image inputs, furthering the goal of Aguirre and Siam. The systems readily integrate, as Aguirre/Siam is also configured to utilize LiDAR (Aguirre, [0003])
With respect to claim 13, Aguirre, Siam, and Beltransen teach:
The state estimation device according to claim 12, wherein the diagnosis unit compares an orientation of the traffic object indicated by the image data, and an orientation of the traffic object calculated based on the installation state parameter estimated by the estimator, and diagnoses the installation state of the imaging device when a degree of coincidence is higher than a determination threshold (Aguirre, Fig. 3A, blocks 316-320 iterating; Aguirre, [0029]; Siam, Fig. 2)
Aguirre/Siam/Beltransen continuously evaluates the sensors’ data against each other –both the received problematic data and the proper data. With reference to Aguirre, Fig. 3A, if a sensor is above a confidence threshold, operations will continue. Under the iteration process, it continues to be diagnosed. As such, it can be said that Aguirre/Siam/Beltransen is configured to “diagnose[] the installation state of the imaging device when a degree of coincidence is higher than a determination threshold” since it is continuously running.
The examiner also understands the language of this claim include a cross-validation mechanism amongst methods. The examiner further admits that generally speaking, Aguirre/Siam/Beltransen determines object orientation in a similar manner across all sensors. However, this distinction was not made explicit within the claim. Such that a reasonable interpretation of the claim may be as follows:
an orientation of the traffic object indicated by the image data: the image analysis pipeline of Aguirre/Siam, with respect to sensor 1
and an orientation of the traffic object calculated based on the installation state parameter estimated by the estimator: for any given sensor 2, the process of Aguirre/Siam, but with the understanding that orientation of a traffic object is also necessarily estimated based on “the installation state parameter” of sensor 2 (a parameter which is ultimately going to be estimated by the estimator, for example, the installation orientation of the sensor, which necessarily will contribute to the determination of the traffic object’s orientation as it captures the image data which is used for calculation. If the sensor was upside down, the orientation of the object would be reversed, and the motion vector of Siam Fig. 2 would be necessarily changed with respect to other sensor outputs)
Accordingly, the language of the claim is met.
Additional References
Additionally cited references (see attached PTO-892) otherwise not relied upon above have been made of record in view of the manner in which they evidence the general state of the art.
The examiner also notes the methods of Sochor et. al Traffic Surveillance Camera Calibration by 3D Model Bounding Box Alignment for Accurate Vehicle Speed Measurement (Hereinafter, “Sochor”), referenced in the information disclosure statement filed 09/04/2025. The reference further evidences the state of the art with regards to automatic object detection using machine-learning techniques for the purpose of automatic camera calibration. Sochor also teaches generally the application of similar object detection auto-calibration techniques to stationary traffic cameras, further evidencing the applicability of methods to the more specific imaging device embodiments of the specification.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NOAH WILLIAM BOYAR whose telephone number is 571-272-8392. The examiner can normally be reached 10:00 – 6:00 EST, Monday – Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NOAH W BOYAR/Examiner, Art Unit 2669
/IAN L LEMIEUX/Primary Examiner, Art Unit 2669
Prosecution Insights