Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 13 and 14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims 13 and 14 claimed a computer program product / a computer-readable storage medium. It can be interpreted as a software / a carrier wave signal. It fails to fall within a statutory category of invention. It is not a process occurring as a result of executing the software, a machine programmed to operate in accordance with the software nor a manufacture structurally and functionally interconnected with the software in a manner which enables the software to act as a computer component and realize its functionality. It is also clearly not directed to a composition of matter. Therefore, it is non-statutory under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1 – 2, 5 – 6, 8 – 9, 11, 13 and 14 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Guo et al. (“Learning to Understand Traffic Signs”, IDS), hereinafter referred as Guo.
Regarding claim 1, Guo discloses a method for detecting and recognizing a street sign in an environment of a motor vehicle by an assistance system of the motor vehicle (abstract; figure 1; page 2076, column 2, lines 3-5), the method comprising:
capturing at least one image of the environment by an optical capturing device of the assistance system (page 2078, column 1, section 3.1: images are from car camera videos);
encoding the captured image by a transformer device of an electronic computing device of the assistance system (figure 4; page 2082, column 1, paragraph 2: the backbone module/stem network corresponds to the encoding);
first decoding of the encoded image by a detection transformer device of the electronic computing device for decoding object features in the captured image (figure 4, module "Head S"; page 2077, column 1, paragraph 1; page 2080, column 1, paragraph 1: the detection head for symbols corresponds to the first decoding);
second decoding of the encoded image, wherein the second decoding is performed in parallel to the first decoding, by a recognition transformer device of the electronic computing device for text recognition in the captured image (figure 4, modules "Head T" and partially "Semantic Description"; page 2080, column 1, paragraph 1; page 2081, column 2, paragraph 2: the detection head for text corresponds to the second decoding and is performed in parallel to the other heads. A part of the text recognition is also performed in the semantic description step.); and
detecting and recognizing of the street sign depending on the decoded object features and the text recognition by the electronic computing device (figure 4; figure 7; page 2081, column 2, paragraph 5: generating the key, value pairs as the semantic description of the traffic sign corresponds to the detecting and recognizing of the street sign).
Regarding claim 2 (depends on claim 1), Guo discloses the method wherein the transformer device is provided as convolutional neural network (figure 4 ("backbone"), on page 2082, column 1, paragraph 2 and on page 2083, column 1, paragraph
1, in the form of the ResNet-50 backbone network, which represents an instance of a convolutional neural network).
Regarding claim 5 (depends on claim 1), Guo discloses the method wherein by the detection transformer device a multiple object detection is performed (Figure 4, texts, symbols and arrowhead).
Regarding claim 6 (depends on claim 1), Guo discloses the method wherein by a Hungarian method and/or a bipartite loss method a joining of the object detection and the text recognition is performed by the electronic computing device (page 2080 equation (4) for a bipartite loss method a joining of the object detection and the text recognition).
Regarding claim 8 (depends on claim 1), Guo discloses the method wherein the recognition transformer device comprising a vocabulary database is provided.
Regarding claim 9 (depends on claim 1), Guo discloses the method wherein the text recognition is performed in a word-based or character-based manner (page 2081, column 2, paragraph 2: the detection head for text corresponds to the second decoding and is performed in parallel to the other heads. A part of the text recognition is also performed in the semantic description step. Also see Figure 2 for recognize word-based or character-based manner).
Regarding claim 11 (depends on claim 1), Guo discloses the method wherein based on the detection and recognition of the street sign an update of an environment map and/or a localization of the motor vehicle (1) relative to the environment is performed (page 2076, column 2, “positioning assistance, and map correction”).
Regarding claims 13 and 14, they are corresponding to claim 1 (with ResNet-50 considered as software in page 2083), thus, they are interpreted and rejected set forth for the same reason for claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 4, 7 and 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Ramachandran et al. (“Stand-Alone Self-Attention in Vision Models”, IDS), hereinafter referred as Ramachandran.
Regarding claim 3 (depends on claim 1), Guo fails to explicitly disclose the method wherein the transformer device comprising a self-attention module is provided.
However, in a similar field of endeavor Ramachandran discloses a method of stand-alone self-attention in vision models (abstract). In addition, Ramachandran discloses the method comprising a self-attention module (section 2.2).
Both Guo and Ramachandran served purpose of image processing. Thus, one of ordinary skill in the art would have been able to make the combination and the outcome would have been predictable to that same person. Combining one for the other achieves the predictable result of allowing achieve gains in image classification and object detection (KSR scenario A).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined a neural network of Guo and for a further comprising a self-attention module of Ramachandran according to known methods to yield the predictable result of achieve gains in image classification and object detection.
Regarding claim 4, it is corresponding to claim 3, thus, it is interpreted and rejected set forth for the same reason for claim 3.
Regarding claim 7 (depends on claim 1), Guo fails to explicitly disclose the method wherein the recognition transformer device comprising a yet further self-attention module is provided.
However, in a similar field of endeavor Ramachandran discloses a method of stand-alone self-attention in vision models (abstract). In addition, Ramachandran discloses the method comprising a yet further self-attention module (section 2.2, local self-attention, global self-attention; computing single-headed attention on each group separately as above with different transforms).
Both Guo and Ramachandran served purpose of image processing. Thus, one of ordinary skill in the art would have been able to make the combination and the outcome would have been predictable to that same person. Combining one for the other achieves the predictable result of allowing achieve gains in image classification and object detection (KSR scenario A).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the neural network of Guo and for a further comprising a yet further self-attention module of Ramachandran according to known methods to yield the predictable result of achieve gains in image classification and object detection.
Regarding claim 10 (depends on claim 1), Guo fails to explicitly disclose the method wherein for text recognition a softmax function of the recognition transformer device is used.
However, in a similar field of endeavor Ramachandran discloses a method of stand-alone self-attention in vision models (abstract). In addition, Ramachandran discloses the method wherein a softmax function of the recognition transformer device is used (section 2.2).
Both Guo and Ramachandran served purpose of image processing. Thus, one of ordinary skill in the art would have been able to make the combination and the outcome would have been predictable to that same person. Combining one for the other achieves the predictable result of allowing achieve gains in image classification and object detection (KSR scenario A).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the text recognition of Guo and for a further comprising a softmax function of the recognition transformer device of Ramachandran according to known methods to yield the predictable result of achieve gains in image classification and object detection.
Claim(s) 12 and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Guo in view of Chen et al. (US Patent Application Publication 2021/0026355, IDS), hereinafter referred as Chen.
Regarding claim 12 (depends on claim 1), Guo fails to explicitly disclose the method wherein for the detection and recognition of the street sign a current speed of the motor vehicle is considered and/or a resolution improvement of the at least one image is performed.
However, in a similar field of endeavor Chen discloses a method of image processing (abstract). In addition, Chen discloses the method wherein for the detection and recognition of an image a current speed of the motor vehicle is considered and/or a resolution improvement of the at least one image is performed ([0121 – 0122]).
Both Guo and Chen served purpose of image processing. Thus, one of ordinary skill in the art would have been able to make the combination and the outcome would have been predictable to that same person. Combining one for the other achieves the predictable result of allowing for more accurately processing the image (KSR scenario A).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined the image processing of Guo and an image a current speed of the motor vehicle is considered of Chen according to known methods to yield the predictable result of allowing for more accurately processing the image.
Regarding claim 15, Guo discloses a method according to claim 1 for a motor vehicle for detecting and recognizing a street sign in an environment of the motor vehicle (see claim 1 for details).
However, Guo fails to explicitly disclose the method is implemented by an assistance system, comprising at least one optical capturing device and an electronic computing device.
However, in a similar field of endeavor Chen discloses a method of image processing (abstract). In addition, Chen discloses the method wherein is implemented by an assistance system, comprising at least one optical capturing device and an electronic computing device (Fig. 1 and 10C).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date to modify the image processing of Guo, and implemented the method by an assistance system, comprising at least one optical capturing device and an electronic computing device of Chen according to known methods to yield the predictable result of allowing the method by a substantial system to put the application in a real use.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QIAN YANG whose telephone number is (571)270-7239. The examiner can normally be reached on Monday-Thursday 8am-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached on 571-270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QIAN YANG/
Primary Examiner, Art Unit 2677
Prosecution Insights