Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-10 are presented for examination
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
The abstract should be in narrative form and generally limited to a single paragraph on a separate sheet within the range of 50 to 150 words in length. The abstract should describe the disclosure sufficiently to assist readers in deciding whether there is a need for consulting the full patent text for details.
The language should be clear and concise and should not repeat information given in the title. It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
The abstract is objected to because in line 1, “Disclosed herein” is used and should be avoided.
The disclosure is objected to because of the following informalities:
Page 11 line 16, recites that “presentation learning is performed for target discrimination,” in which “presentation learning” appears to be a typographical error for “representation learning,” the term used consistently throughout the remainder of the disclosure (see, for example, the same paragraph on page 11 line 19 reciting “representation learning for goals,” and page 12 line 4 and page 19 line 2)
Page 23 line 2, recites setting an instruction for “an under-exploded target,” in which “under-exploded” appears to be a typographical error for “under-explored,” the term used throughout the remainder of the disclosure (see, for example, page 3 line 11, page 11 line 9, and page 18 line 2)
Page 34 line 8 and page 35 line 4 recite, in three places, “the embodiment descried through the present specification,” in which “descried” appears to be a typographical error for “described”
Page 29 line 11, in the description of step S620, refers to “status information for the target,” whereas this same information is referred to as “state information” throughout the remainder of the disclosure, in the drawings (FIG. 6), and in the claims (see, for example, page 5 line 22, page 14 line 8, and page 15 line 8, and claim 5 line 4), and the inconsistent terminology should be reconciled
Appropriate correction is required.
Claim Objections
Claim 1 is objected to because of the following informalities: claim 1 recites " a controller provided with at least one process," in which "process" appears to be a typographical error for "processor." The specification describes the controller as including "at least one
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, lines 15-18, it is not clearly understood what is meant by “train a reinforcement learning-based learning model configured to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set”. It is not clear whether the claim intended to recite “train a reinforcement learning-based learning model to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set” or “train a reinforcement learning-based model, wherein the reinforcement learning-based learning model is configured to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set”. For the purpose of examination, the examiner will interpret the limitation as “train a reinforcement learning-based learning model to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set.”
Regarding claim 5, lines 8-11, it is not clearly understood what is meant by “training a reinforcement learning-based learning model configured to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set”. It is not clear whether the claim intended to recite “training a reinforcement learning-based learning model to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set” or “training a reinforcement learning-based model, wherein the reinforcement learning-based learning model is configured to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set”. For the purpose of examiner, the examiner will interpret the limitation as “training a reinforcement learning-based learning model to receive the instruction for the target and the state information for the target and output action information by referring to the instruction-target set.”
Regarding claims 2-4 and 6-10, claims 1-4 and 6-10 are also rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for depending on an indefinite parent claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-7, and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Qi et al. (CN 110727346 B, filed 09/24/2019), hereinafter Qi, and in view of Zhang et al. (KR 102345267 B1, filed 10/12/2020), hereinafter Zhang.
Regarding claim 1, Qi teaches: A multi-target analysis apparatus comprising: (Qi; Abstract, Briefly described, a device with the ability to obtain a set of command targets corresponding to a command intent of voice commands):
an input/output interface configured to receive data and output results of computation
of the data: (Qi; Page 11 Paragraph 80, Briefly described, an input and output device; Page 11 Paragraph 83, Briefly described, the input device can receive data such as digital or character information, and the output device includes display devices, speakers, and audio devices to output results of computation of the data):
storage configured to: (Qi; Page 11 Paragraph 80, Briefly described, a memory;
Page 11 Paragraph 82, Briefly described, the memory including a data storage area and non-volatile memory, such as a solid-state storage device):
store a program for performing a multi-target analysis method: (Qi; Page 11, Paragraph
82, Briefly described, the memory including a program storage area to perform a multi-target analysis method):
a controller provided with at least one process, and configured to analyze multiple
targets received through the input/output interface by executing the program: (Qi; Abstract, Briefly described, obtaining a set of command targets; Page 11 Paragraph 80, Briefly described, a processor and an input and output device; Page 11 Paragraph 81, Briefly described, the processor able to execute various functional applications and data processing by running software programs; Page 11 Paragraph 83, Briefly described, receiving targets as digital or character information):
wherein the controller is further configured to: collect instruction-target pairs in each of
which an instruction and state information for a target are matched with each other so that the target is specified through the instruction, and generate an instruction-target set having a plurality of instruction-target pairs: (Qi; Page 6, Paragraph 39, Briefly described, an instruction determined through a captured voice command, and a target collected through the user’s multimodal information to form matched pairs, where these instruction target pairs form an instruction target set that can have these pairs pulled from; Page 7 Paragraph 43, Briefly described, signals to activate a working state over a standby state of the vehicle to receive commands and collect the user’s multimodal information for instruction targets in the working state; Page 7 Paragraph 48, Briefly described, a corresponding set of instruction targets with at least one instruction target for a provided instruction intent to conform to; Page 7, Paragraph 49, Briefly described, instructions without a target In the command can have their target determined by the content of the command, for example, the command “listen to music” to the vehicle determines that the target is inside the vehicle; Page 11 Paragraph 83, Briefly described, the generation of signal inputs for creating new states of the vehicle after a command is received leading to a generation of a new instruction target set containing instruction target pairs):
output action information by referring to the instruction-target set: (Qi; Page 6
Paragraph 39, Briefly discussed, the instruction target set; Page 7 Paragraph 43, Briefly discussed, an action output by the vehicle to enter a certain state after receiving a voice command to provide an instruction target; Page 7 Paragraph 56, Briefly described, a feedback confirmation to the user is provided to determine if the output action fulfilled the user’s needs).
However, Qi fails to expressly teach – train a reinforcement learning-based learning model configured to receive the instruction for the target and the state information for the target.
In the same field of endeavor, Zhang teaches:
train a reinforcement learning-based learning model configured to receive the
instruction for the target and the state information for the target: (Zhang; Page 3 Paragraph 9, Briefly described, a reinforcement learning model; Page 4 Paragraph 2, Briefly described, the feature extraction unit, a part of the reinforcement learning model, can receive an instruction for a target and the state information for a target).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated – train a reinforcement learning-based learning model configured to receive the instruction for the target and the state information for the target as suggested by Qi and Zhang. Doing so would be desirable because a reinforcement model can be used to determine the target and output action information by referring to an instruction-target set. The combined method would provide the technical architecture for multimodal machine learning and a reinforcement model to address the multimodal targets the model receives through instruction and state information. The targets in which the reinforcement model can receive an instruction and state information can be environmental from the combination of Qi and Zhang, so the multimodal machine learning method using sensory inputs like voice commands can be a form of reinforcement learning.
Regarding claim 2, the combination of Qi and Zhang teaches the invention as claimed in claim 1 above including wherein the reinforcement learning-based learning model includes: a feature extraction model configured to receive the state information for the target and output the state feature information: (Zhang; Page 3 Paragraph 7, Briefly described, a reinforcement learning model including a feature extraction model; Page 3 Paragraph 8, Briefly described, the feature extraction model capable of receiving state information in the form of features that indicate the state and the target, and outputting based on the features extracted on the state data):
and a reinforcement learning-based learning model connected to the feature extraction
model, and configured to receive the instruction for the target and the state feature information and output the action information: (Zhang; Page 3 Paragraph 7, Briefly described, a reinforcement learning model including a feature extraction model; Page 3 Paragraph 8, Briefly described, the feature extraction model capable of receiving state information in the form of features that indicate the state and the target, and outputting actions based on the features extracted on the state data; Page 4 Paragraph 2, Briefly described, the reinforcement learning model capable of receiving an instruction for a target).
Regarding claim 3, the combination of Qi and Zhang teaches the invention as claimed in claim 2 above including wherein the controller applies a method of measuring a success rate of the target in an update process according to an episode of reinforcement learning and then adjusting a sampling rate of a target to be focused on learning based on the success rate, utilizes instructions, stored in the instruction-target set in the feature extraction model, as labels of the feature extraction model, and increases an amount of training data for the target as a degree of change in the success rate increases: (Qi; Page 6 Paragraph 39, Briefly described, instruction target pairs and an instruction target model, where instructions are paired and stored with targets into instruction target pairs; Page 11 Paragraph 83, Briefly described, signal inputs are generated for new instruction target set generation for an increase in the amount of training data) (Zhang; Page 3 Paragraph 8, Briefly described, a feature extraction model; Page 3 Paragraph 9, Briefly described, a reinforcement learning model for reinforcement learning; Page 4 Paragraph 2, Briefly described, the feature extraction model utilizing the instructions stored as labels for a target and a state; Page 4 Paragraph 10, Briefly described, measuring the success rate of a target based on image collection of target data (target image) whenever the reinforcement learning succeeds in achieving the goal and labels the collected target data corresponding to the target; Page 4 Paragraph 11, Briefly described, the reinforcement learning model collects data related to an episode of reinforcement learning (reaching a target state) to update and adjust the sampling rate to be focused learning based on the success rate, and after labeling the collected target data, stores it in the target storage for further training as the success rate increases).
Regarding claim 5, Qi teaches: the multi-target analysis method comprising: (Qi; Abstract, Briefly described, a method with the ability to obtain a set of command targets corresponding to a command intent of voice commands):
collecting instruction-target pairs in each of which an instruction and state information
for a target are matched with each other so that the target is specified through the instruction, and storing an instruction-target set having a plurality of instruction-target pairs: (Qi; Page 6, Paragraph 39, Briefly described, an instruction determined through a captured voice command, and a target collected through the user’s multimodal information to form matched pairs, where these instruction target pairs form an instruction target set that can have these pairs pulled from, and instruction target sets that are stored and containing a plurality of instruction target pairs; Page 7 Paragraph 43, Briefly described, signals to activate a working state over a standby state of the vehicle to receive commands and collect the user’s multimodal information for instruction targets in the working state; Page 7 Paragraph 48, Briefly described, a corresponding set of instruction targets with at least one instruction target for a provided instruction intent to conform to; Page 7, Paragraph 49, Briefly described, instructions without a target In the command can have their target determined by the content of the command, for example, the command “listen to music” to the vehicle determines that the target is inside the vehicle):
output action information by referring to the instruction-target set: (Qi; Page 6
Paragraph 39, Briefly discussed, the instruction target set; Page 7 Paragraph 43, Briefly discussed, an action output by the vehicle to enter a certain state after receiving a voice command to provide an instruction target; Page 7 Paragraph 56, Briefly described, a feedback confirmation to the user is provided to determine if the output action fulfilled the user’s needs).
However, Qi fails to expressly teach – training a reinforcement learning-based learning model configured to receive the instruction for the target and the state information for the target.
In the same field of endeavor, Zhang teaches:
training a reinforcement learning-based learning model configured to receive the
instruction for the target and the state information for the target: (Zhang; Page 3 Paragraph 9, Briefly described, a reinforcement learning model; Page 4 Paragraph 2, Briefly described, the feature extraction unit, a part of the reinforcement learning model, is able to receive an instruction for a target and the state information for a target).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated – train a reinforcement learning-based learning model configured to receive the instruction for the target and the state information for the target as suggested by Qi and Zhang. Doing so would be desirable because a reinforcement model can be used to determine the target and output action information by referring to an instruction-target set. The combined method would provide the technical architecture for multimodal machine learning and a reinforcement model to address the multimodal targets the model receives through instruction and state information. The targets in which the reinforcement model can receive an instruction and state information can be environmental from the combination of Qi and Zhang, so the multimodal machine learning method using sensory inputs like voice commands can be a form of reinforcement learning.
Regarding claim 9, it is computer-readable storage medium claim that corresponds to the method of claim 5 above. Therefore, it is rejected for the same reason as method claim 5 above.
Regarding claim 10, it is product claim that corresponds to the method of claim 5 above. Therefore, it is rejected for the same reason as method claim 5 above.
Regarding claims 6 and 7, they are method claims that correspond to apparatus claims 2 and 3. Therefore, they are rejected for the same reason as claims 2 and 3 above.
Claims 4 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Qi in view of Zhang, as applied in the rejection of claims 2 and 6 above, further in view of Kim et al. (KR 102122232 B1, filed 12/31/2019), hereinafter Kim.
Regarding claim 4, the combination of Qi and Zhang teaches the invention as claimed in claim 2 above wherein the controller applies a method of adjusting the instructions in a process of performing reinforcement learning: (Zhang; Page 3 Paragraph 9, Briefly described, a reinforcement learning model for reinforcement learning; Page 4 Paragraph 10, Briefly described, target data stored as using instructions as labels for where they can be adjusted to based on the reinforcement learning on the targets).
of such targets stored in the instruction-target set in the reinforcement learning model:
(Qi; Page 6 Paragraph 39, Briefly described, an instruction target set) (Zhang; Page 4 Paragraph 2, Briefly described, a reinforcement learning model with the ability to receive instructions).
However, the combination of Qi and Zhang fail to expressly teach – increases a number of explorations for a target requiring learning by setting, based on a proportion of a number; and – the instructions in inverse proportion to the proportion.
In the same field of endeavor, Kim teaches:
increases a number of explorations for a target requiring learning by setting, based on a
proportion of a number: (Kim; Page 4 Paragraph 3, Briefly described, a reinforcement learning model storing state and action information as numbers to explore in layers; Page 7 Paragraph 4, Briefly described, an increase of a number of explorations through balancing):
the instructions in inverse proportion to the proportion: (Kim; Page 4 Paragraph 3, Briefly
described, parameter information mirroring the understanding of an instruction; Page 7 Paragraph 4, Briefly described, balancing done by taking the instructions and inversing them proportional to the number of visits of the original proportion of visited action pairs).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated – increases a number of explorations for a target requiring learning by setting, based on a proportion of a number; and – the instructions in inverse proportion to the proportion as suggested by Qi, Zhang, and Kim. Doing so would be desirable because the reinforcement learning model can be trained to better exploration of the instruction target sets by having the instructions be in inverse proportion to the proportion from the combination of Qi, Zhang, and Kim. The combined method would provide the ability for the reinforcement learning model to adjust the instructions according to their success in reaching a target utilizing their instructions, and then account for under-analyzed targets by inverting the instructions to account for further exploration.
Regarding claim 8, it is a method claim that corresponds to apparatus claim 4. Therefore, it is rejected for the same reason as claim 4 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Wang et al. (CN 115186163 A) teaches in Page 18 Paragraph 3 an input/output interfaced configured to receive data and output results of the computation of the data and a storage; Contents of the Invention Paragraph 1, a multi-target analysis apparatus.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROLANDO PATRICK VIRREIRA whose telephone number is (571)270-1570. The examiner can normally be reaches Monday – Friday, 8:30AM-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at
http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached on (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions. Contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROLANDO PATRICK VIRREIRA/Examiner, Art Unit 2143
/JENNIFER N WELCH/Supervisory Patent Examiner, Art Unit 2143
Prosecution Insights