Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status of the Application
Claims 1-14 have been examined in this application. This communication is a Non Final Office Action on the on merits. The Information Disclosure Statement (IDS) filed on 8/25/2025 has been acknowledged by the Office.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
A series of singular dependent claims is permissible in which a dependent claim refers to a preceding claim which, in turn, refers to another preceding claim.
A claim which depends from a dependent claim should not be separated by any claim which does not also depend from said dependent claim. It should be kept in mind that a dependent claim may refer to any preceding independent claim. In general, applicant's sequence will not be changed. See MPEP § 608.01(n).
Specifically, claim 6 depends on claim 2 but is separated from claim 2 by claims 3-5.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f), except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: recognition unit, association definition unit, control processing configuration unit, processing optimization unit, storage unit, and operation handle in claims 1, 5, and 13.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof, see FIG. 2, as well as FIG. 4 and P [0028]-[0029] which associates the recognition unit, association definition unit, control processing configuration unit, and processing optimization unit with a CPU as part of a controller. See also memory 14 as a solid state drive for the storage unit.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
Claims 7-11 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 7 recites the limitation "each operation unit" in regards to parameters set for imaging processing involved in a control processing scheme. There is insufficient antecedent basis for this limitation in the claim. Claim 6 establishes the use of “operation units” but claim 7 does not depend on claim 6.
Claims 8-11 rejected for their dependency on a rejected claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2 and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizobe et al., hereinafter Mizobe (Document ID: US 20220126440 A1) in view of Tawara (Document ID: US 20220016784 A1).
Regarding claims 1 and 14, Mizobe teaches a work process support system and a work process support method cause a work process support system to execute steps comprising:
a recognition unit that acquires work location information that is information regarding a work location of a robot, and work object information that is information of a work object (see at least detecting section 42 which “detects a target object and specifies a position and a posture of the target object” in P [0039]. See also FIG. 1 wherein the imaging section 21 and illuminating section 22 define the optical system 20 and (P [0039]) and capture information about the workbench, which is the work location of a robot);
Mizobe additionally teaches beginning in P [0077] and FIG. 6 the use of a setting device “for specifying the operations of the robots”. In P [0031] the imaging section 21 is used to “capture an image in a visual field”. Finally, in P [0072] the “initial parameters” for creating a work process of the robot are defined. The configuration description of Mizobe does not explicitly describe
an association definition unit that, when a work process of the robot is created, associates the work process with the work location information and the work object information;
Instead, Tawara, whose invention pertains to easing user operation of a robot program, teaches a processor 300A as part of a work process support system, and in at least FIG. 10 a process for registering a hand model for a work process in accordance with the work location--- as seen in P [0144] through registering the start position and placing position--- and the work object information--- see at least P [0137] for the step SC5 for registering the workpiece to be held by the hand. In P [0139] it is clear that the registration process is meant to define “A plurality of possible holding positions held by the robot RBT” for a work process.
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to have modified the setting device, camera, and work process definition of Mizobe with the registration of robot properties, work location, and object information for a work holding and conveying process of Tawara in order to use robot data, work location information, and work object information to develop a versatile robot program, as disclosed in P [0013] of Tawara, wherein a robot program, or work process, is intended to be usable across a number of different circumstances. `
In view of the modification, Mizobe further teaches
a control processing configuration unit that determines a configuration of control processing for the robot based on the association by the association definition unit and calculates a computation amount in the control processing (see at least P [0077] for the beginning of “configuration of the setting device” which is used to realize the operations for control processing for the robot. In general, in P [0080] “It is assumed that an operation for moving the target object W held by the gripper 23 of the robot 3 from a certain point to another point is determined as an optimizing operation (see FIG. 1).”, which builds on the modified association above. Then, in P [0081] “the initial parameters 44p [are] received from the control device 40”, and an “operation time” as a computation amount is calculated in P [0085]); and
a processing optimization unit that determines, based on the computation amount and a priority item selected by a user, assignment of processing to a computing resource for processing to be performed in the work process and outputs a result of the assignment (see at least FIGs. 10-11 and P [0102]-[0116] wherein input from a user defines a priority item, and “the optimization processing for the parameter set can be efficiently performed”. Note that the optimization process is highly flexible, and the operation time, or computation amount, can be set as a restriction as well, as seen in FIG. 15.).
Regarding claim 2, modified Mizobe teaches the work process support system according to claim 1, and Mizobe further teaches
a GUI screen that is a user interface is displayed on a display device (see user interface beginning in FIG. 10), and
a series of operations to be performed by the robot is set by the user operating the GUI screen (see again from FIG. 10 onwards wherein a series of operations to be performed by the robot is set by the user operating the GUI screen based on optimized parameters).
Regarding claim 13, modified Mizobe teaches the work process support system according to claim 1, and Mizobe further teaches
the robot includes a robot arm and a robot hand (see at lest FIG. 1 and P [0028] wherein the robot includes an arm and an end effector gripper 23 as a robot hand),
the work process support system further comprises:
an imaging device (imaging section 21);
a controller that controls the robot arm and the robot hand (control device 40);
Mizobe teaches the use of a display device 602 as a user interface and display screen for “evaluating a control result of the robot 3”. But Mizobe does not explicitly teach the use of
an operation handle for operating the robot arm and the robot hand; and
a display device that displays an image captured by the imaging device, and
the robot arm and the robot hand operate based on an input via the operation handle.
Tawara does teach the use of a display device that displays an image captured by the imaging device in at least FIG. 12 wherein the display is shown side by side with the edit screen of a robot arm. Additionally, in P [0138] Tawara teaches “directly operating the hand model with a mouse of the control panel 4”, wherein the mouse acts as an operation handle for direct control of the robot arm and hand model. In P [0141] actual robot motion is executed based on the registered holding position specified by the operation handle.
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to have modified the display system for parameter robot control and robot control of Mizobe with the direct input for teaching position of control on a display of Tawara in order to simplify the definition of robot holding positions for a gripping and moving operation of a robotic arm as in P [0140] of Tawara.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizobe in view of Tawara, and further in view of Ebrahimi Afrouzi et al., hereinafter Ebrahimi Afrouzi (Document ID: US2022187841A1).
Regarding claim 3, modified Mizobe teaches the work process support system according to claim 1, and Mizobe teaches in P [0031] the use of an imaging section capable of adjusting brightness in conjunction with an illuminating section 22. Tawara teaches in FIG. 7 a light projecting part that works with a sensor to irradiate an object and its surroundings with light. But Mizobe and Tawara do not explicitly teach that
the recognition unit recognizes and quantifies, based on an image of the work location captured by an imaging device, information regarding brightness of the work location and information regarding a dominant color of the work location, and stores the quantified results to the work location information.
Instead Ebrahimi Afrouzi, whose invention pertains to operating a wheeled device to perform work in an environment, teaches in P [1062] the ability to “detect and localize boundaries in scenes using local image measurements”, which is based on an algorithm that can “respond to changes in brightness, color and texture.” Then, in P [0981] there is disclosure to support the ability to “localize the robot by localizing against the dominant color in each area.”
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to have modified the ability to change imaging properties of an optical system including brightness of Mizobe and Tawara with the image based localization using brightness of a location and a dominant color of Ebrahimi Afrouzi in order to utilize known methods of robotic image processing and color considerations to identify objects within an environment and partition a scene detected by a robot, as in P [0980]-[0981] of Ebrahimi Afrouzi.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Mizobe in view of Tawara, and further in view of Takacs et al., hereinafter Takacs (NPL Reference: Control of Robotic Arm with Visual System).
Regarding claim 4, modified Mizobe teaches the work process support system according to claim 1, and Tawara teaches in P [0102]-[0104] the ability to capture and store “three-dimensional shape data representing the three-dimensional shape of each of the workpieces WK stowed in bulk” as an image characteristic amount of the work object, which is further seen through the known methods of “a pattern projection method, a stereo method, a shape-from-focus method, a light-section method, an optical radar method, an interferometry method (white interferometry method), a TOF method, and the like,”. But Mizobe and Tawara do not explicitly teach that the recognition unit recognizes a color value and stores the color value in addition to the image characteristic amount of the work object.
Instead, Takacs, whose work pertains to sorting objects by color, shape, and combination, teaches in at least section “E. Colour-based Object Search Method” the ability to distinguish and record color values for objects in addition to their shape and patterns.
It would have been obvious to one of ordinary skill in the art before the filing date of the claimed invention to have modified the image characteristic recording of Mizobe and Tawara with the colour value sorting and storage for objects of Takacs in order to utilize known methods of robotic image processing and color considerations to identify and sort objects based on shape and color.
Allowable Subject Matter
Claims 7-11 have not had art applied in this Office Action, but are not considered potentially allowable in view of the rejections under 35 U.S.C. 112(b). Claims objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Document ID: US 20120072023 A1
Invention pertains to registering a target object for use of robot control.
Document ID: US 20220414398 A1
Invention pertains to an item identification model and training a dataset for item identification.
Document ID: US 20210283782 A1
Invention pertains to object recognition and parameter optimization for a robot system.
Document ID: US 20220366651 A1
Invention pertains to generating key points from captured images.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Dairon Estevez whose telephone number is (703)756-4552. The examiner can normally be reached M-F 8:00AM - 4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Khoi Tran can be reached at (571) 272-6919. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/D.E./Examiner, Art Unit 3656
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights