DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1 and 3-5 have been amended. Claims 6-8 have been canceled. Claims 9-23 have been added. Claims 1-5 and 9-23 remain pending and have been examined.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 5, 10, 15 and 19 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent Application Publication 20190080205 by Kaufhold et al. ("Kaufhold").
In regard to claim 1, Kaufhold discloses:
1. A generalized data generation device comprising circuitry configured to execute a method comprising: See Kaufhold, Fig. 26, depicting a device for method execution.
training a generalized model for training for obtaining data satisfying a general parameter through predetermined machine learning by using a general training dataset as input, the general training dataset being a set of data satisfying the general parameter from among multiple types of parameters, and Kaufhold, Fig. 4 and ¶ 0098, “The original training images 411 … are used to train the object recognizer 415.” Also see ¶ 0085, broadly describing multiple types of parameters/classes/categories, e.g. “objects are identified as belonging to a specific category.” … outputs a trained generalized model; and Kaufhold Fig. 6 and ¶ 0096, “The translator 609 is then trained to produce a network that can then be called upon with any unpaired synthetic image to produce a translated version of the image.”
generating a generalized input dataset generalized by using an input dataset, the input dataset being a set of data satisfying any of the multiple types of parameters, and the trained generalized model, such that the input dataset satisfies the general parameter. Kaufhold ¶ 0098, “The original training images 411 and translated images 413 created by 401 are used to train the object recognizer 415. The results of performing object recognition on an unknown data set are shown on graph 417, which plots the objects recognized given the number of training examples and the accuracy. Points 419 show that the object recognizer 415 is able to achieve high accuracy rates with few real-world (original) training examples because the object recognizer 415 was trained with supplemental translated images 413.”
In regard to claim 3, Kaufhold discloses:
3. An estimation device comprising circuitry configured to execute a method comprising: See Kaufhold, Fig. 26, depicting a device for method execution.
training a state estimation model for training for estimating the state of a target object through predetermined machine learning by using a general training dataset as input, the general training dataset being a set of data satisfying a general parameter from among multiple types of parameters, and outputs a trained state estimation model; and Kaufhold ¶ 0098, “The original training images 411 … are used to train the object recognizer 415.” Also see Fig. 4, depicting a general training dataset 411 used as input for training. Also see ¶ 0085, broadly describing multiple types of parameters/classes/categories, e.g. “objects are identified as belonging to a specific category.” Also see ¶ 0095, “the object recognizer 1105 which is a deep neural network.”
estimating the state of the target object by using a generalized input dataset and the trained state estimation model, Kaufhold, Fig. 4, depicting state estimation using object recognizer 415 and generalized dataset 413.
… the generalized input dataset being generalized by using an input dataset, the input dataset being a set of data satisfying any of the multiple types of parameters, and a trained generalized model obtained by performing machine learning on the general training dataset, such that the input dataset satisfies the general parameter. Kaufhold, Fig. 4, depicting generalized dataset 413. Fig. 6 and ¶ 0096, “The basic process flow for training the translator, as shown in FIG. 6 is for a set of synthetic images 603 to be paired 605 with training images 601. The translator 609 is then trained to produce a network that can then be called upon with any unpaired synthetic image to produce a translated version of the image.” Also ¶ 0120, “Typically, a set of training images for a given object (or an object class), such as training images … 411, … 601, … will comprise a relatively sparse set of real-world images.”
In regard to claim 5, Kaufhold discloses:
5. A generalized data generation method comprising: See Kaufhold, Figs. 4 and 6, depicting data generation methods.
All further limitations of claim 5 have been addressed in the above rejection of claim 1.
In regard to claim 10, Kaufhold discloses:
10. The generalized data generation device according to claim 1, wherein the generalized model includes a machine learning model. Kaufhold, ¶ 0095, “the object recognizer 1105 which is a deep neural network …”
In regard to claim 15, Kaufhold also discloses:
15. The estimation device according to claim 3, wherein the state estimation model includes a machine learning model. Kaufhold, ¶ 0095, “the object recognizer 1105 which is a deep neural network …”
In regard to claim 19, parent claim 5 is addressed above.
All further limitations of claim 19 have been addressed in the above rejection of claim 10.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 4, 11-13, 17-18 and 21-23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kaufhold as applied above, and further in view of U.S. Patent Application Publication 20200198623 by Günzel ("Günzel").
In regard to claim 2, Kaufhold also discloses:
2. The generalized data generation device according to claim 1, wherein the data satisfying the general parameter is … [a first class], and the input dataset includes … [a second class]. Kaufhold, ¶ 0093, “The translation can be applied to a set of images from one domain to be translated to a different domain as shown in FIG. 15 where drawings of cats 1501 can be translated 1503 as photo realistic images of cats 1505. The translation can be applied to a set of images of a particular class translated to a set of images of another class.”
Kaufhold does not expressly disclose: road surface data expressing a smooth road surface … road surface data expressing a rough road surface. This is taught by Günzel. See Günzel, ¶ 0023, “the surface of the roadway runs flat, that is, it has no road bumps, such as ground-level obstacles.” ¶ 0030, “the surface condition of the ground-level obstacles, such as surface roughness.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Günzel’s images with Kaufhold’s generalized model in order to automatically operate a motor vehicle efficiently and smoothly when driving as suggested by Günzel (see ¶ 0039).
In regard to claim 4, Kaufhold does not expressly disclose the claimed limitations.
However, these are taught by Günzel:
4. The estimation device according to claim 3, wherein the target object is a road surface, and the circuitry further configured to execute a method comprising: estimating one of a state in which the road surface is flat, Günzel ¶ 0023, “flat” … a state in which the road surface has a level difference, and Günzel ¶ 0023, “height differences” … a state in which the road surface is inclined. Günzel ¶ 0023, “inclination.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Günzel’s road conditions with Kaufhold’s generalized model in order to automatically operate a motor vehicle efficiently and smoothly when driving as suggested by Günzel (see ¶ 0039).
In regard to claim 11, Kaufhold also discloses:
11. The generalized data generation device according to claim 1, wherein the trained generalized model includes a model obtained by machine learning as an autoencoder, the autoencoder compressing and reconstructing … [data]. Kaufhold, ¶ 0034-0035, “Formally, within an autoencoder, a function maps input data to a hidden representation using a non-linear activation function. This is known as the encoding: … A second function may be used to map the hidden representation to a reconstruction of the expected output. This is known as the decoding: …” Also ¶ 0098, “DMTG 401 includes a translator 403 comprising an autoencoder 405 …”
Kaufhold does not expressly disclose: road surface data expressing a smooth road surface. This is taught by Günzel. See Günzel, ¶ 0023, “the surface of the roadway runs flat, that is, it has no road bumps, such as ground-level obstacles.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Günzel’s images with Kaufhold’s generalized model in order to automatically operate a motor vehicle efficiently and smoothly when driving as suggested by Günzel (see ¶ 0039).
In regard to claim 12, Kaufhold and Günzel also teach:
12. The generalized data generation device according to claim 2, wherein the multiple types of parameters include the smooth road surface and the rough road surface. See Günzel, ¶ 0023, “the surface of the roadway runs flat, that is, it has no road bumps, such as ground-level obstacles.” ¶ 0030, “the surface condition of the ground-level obstacles, such as surface roughness.”
In regard to claim 13, parent claim 3 is addressed above.
All further limitations of claim 13 have been addressed in the above rejection of claim 2.
In regard to claim 17, Kaufhold discloses:
17. The estimation device according to claim 3, wherein the generalized model for training includes a model obtained by machine learning as an autoencoder, the autoencoder compressing and reconstructing … [data]. Kaufhold, ¶ 0034-0035, “Formally, within an autoencoder, a function maps input data to a hidden representation using a non-linear activation function. This is known as the encoding: … A second function may be used to map the hidden representation to a reconstruction of the expected output. This is known as the decoding: …” Also ¶ 0098, “DMTG 401 includes a translator 403 comprising an autoencoder 405 …”
All further limitations of claim 17 have been addressed in the above rejection of claim 2.
In regard to claims 18, 21 and 23, parent claim 5 is addressed above.
All further limitations of claims 18 and 21 have been addressed in the above rejections of claims 2 and 11-12, respectively.
In regard to claim 22, parent claim 13 is addressed above.
All further limitations of claim 22 have been addressed in the above rejection of claim 12.
Claim(s) 9, 14, 16, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kaufhold as applied above, and further in view of U.S. Patent Application Publication 20200234088 by Taha et al. ("Taha").
In regard to claim 9, Kaufhold also discloses:
9. The generalized data generation device according to claim 1, wherein the generalized model includes a … neural network. Kaufhold, ¶ 0095, “the object recognizer 1105 which is a deep neural network …” Kaufhold does not expressly disclose: convolutional. Taha teaches this. Taha, ¶ 0002, “… Convolutional Neural Networks (CNN) have been employed with software to help efficiently classify traffic participants and traffic situations images.” It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to use Taha’s CNN with Kaufhold’s object recognizer in order to help efficiently classify traffic participants and traffic situations images as suggested by Taha.
In regard to claim 14, Kaufhold discloses:
14. The estimation device according to claim 3, wherein the state estimation model includes a … neural network. Kaufhold, ¶ 0104, “the object recognizer 1017 trains its own deep network.”
All further limitations of claim 14 have been addressed in the above rejection of claim 9.
In regard to claim 16, parent claim 3 is addressed above
All further limitations of claim 16 have been addressed in the above rejection of claim 9.
In regard to claim 20, parent claim 5 is addressed above.
All further limitations of claim 20 have been addressed in the above rejection of claim 9.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Patent 7398154 to Phuyal et al. See col. 14, lines 7-9, “For example, a positive slope indicates an incline, a slope of zero represents a flat section, and a negative slope indicates a decline.”
U.S. Patent Application Publication 20200238999 by Batts et al. See ¶ 0129, “As shown in FIG. 8, a flat object, such as the ground 802, will reflect light 804a-d emitted from a LiDAR system 602 in a consistent manner. … However, if an object 808 obstructs the road, light 804e-f emitted by the LiDAR system 602 will be reflected from points 810a-b in a manner inconsistent with the expected consistent manner. From this information, the AV 100 can determine that the object 808 is present.” Also ¶ 0209, e.g. “neural networks.”
U.S. Patent Application Publication 20200184260 by Lai et al. See ¶ 0025, “… execute … a plurality of stages of deep convolution layer algorithm for each of sensor to generate a feature result to perform a detection prediction. Through the provided framework, the feature result could be extracted to perform the detection prediction so as to improve an instantaneous calculating speed and reduce unnecessary amount of data.”
U.S. Patent Application Publication 20210052049 by Zheng et al. See Abstract, e.g. “The luggage protection system further includes a ground detection unit attached to the luggage body. The ground detection unit is configured to determine whether the ground adjacent to the luggage body is substantially level.”
Ros et al. "The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes," See Abstract, “In order to answer this question we have generated a synthetic collection of diverse urban images, named SYNTHIA, with automatically generated class annotations. We use SYNTHIA in combination with publicly available real-world urban images with manually provided annotations. Then, we conduct experiments with DCNNs that show how the inclusion of SYNTHIA in the training stage significantly improves performance on the semantic segmentation task.”
Mayer et al. “What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?” See Abstract, “The dominant data acquisition method in visual recognition is based on web data and manual annotation. Yet, for many computer vision problems, such as stereo or optical flow estimation, this approach is not feasible because humans cannot manually enter a pixel-accurate flow field. In this paper, we promote the use of synthetically generated data for the purpose of training deep networks on such tasks.”
Inoue et al. “Transfer learning from synthetic to real images using variational autoencoders for robotic applications” See Abstract, “We thus propose a method that transfers learned capability of detecting object position from a simulation environment to the real world. Our method enables us to use only a very limited dataset of real images while leveraging a large dataset of synthetic images using multiple variational autoencoders.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James D Rutten whose telephone number is (571)272-3703. The examiner can normally be reached M-F 9:00-5:30 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen can be reached at (571)272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/James D. Rutten/Primary Examiner, Art Unit 2121