DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
2. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Status
3. Claims 1-9 are pending in this application.
Specification
4. The specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification. MPEP § 608.01.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
5. Claim 5 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The claim recites, “wherein the hardware processor is configured to set the holding position at a substantial surface gravity center or a substantial center of the identification region.” It appears to be impossible for image analysis as performed by the applicant’s invention to determine the surface gravity of an object, and moreover the concept of a center of surface gravity is strange on its face as a center of gravity is always within a concave object such as a typical box-shaped parcel or container and not on its surface. Furthermore, surface gravity is a term of art in astrophysics, referring to such objects as planets, stars, and asteroids, and is not generally meaningful for parcels and containers in an article handling application. While the examiner may suspect the applicant intended “center of gravity” rather than “surface gravity center” this conclusion cannot be fairly drawn, and so the term must be regarded as indefinite. For subsequent interpretation of the claim in this action (see rejection under 35 U.S.C. 103 below), we consider “substantial surface gravity center” to be “substantially at the center” due to the indefiniteness of “surface gravity”.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
6. Claims 1, 3, and 7-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Liu, et al., CN 107597600 (hereinafter Liu).
7. Regarding claim 1,
Liu discloses:
An information processing device (100: fig. 1) comprising: a connection device (105: fig. 1) configured to connect to an imaging device (image recognition device 102: fig. 1), the imaging device imaging a region on which an object to be handled by using a load handling device (103: fig. 1) is placed;Any photographic image inherently defines a region of its subject. Liu discloses the connection to a load-handling device (robotic arm) and its use in [0016]-[0017]. Liu’s image recognition device 102 is a system and not a unitary machine that per [0035]-[0036] comprises devices such as cameras and image sensors as well as a computer with processor and a network interface per [0009] because in addition to merely acquiring images, the image recognition device can also extract barcodes, determine destination information from barcodes, and perform other computational tasks that require a processor.
and a hardware processor (component of image recognition device 102: fig. 1) connected to the connection device and configured to acquire image information about the region via the connection unit device, detect, from the image information, an identification region in which identification information attached to the object is included, and recognize the identification information in the identification region, output region information related to the identification region and output the identification information.We consider that Liu’s image recognition device 102 comprises a hardware processor, because a) all networked devices comprise processors and b) image recognition and barcode extraction is a computerized function that requires a processor. According to Liu’s [0036] and [0037] along with fig. 1, the image recognition device 102, robot arm controller 104, and server 106 all interact via the network 105. Because all device interaction is on the network, the acquisition of image information occurs on the network too (“via the connection device”), as claimed. Per [0009] and [0078] the controller a) obtains barcode information from the imaging device and b) grasps packages according to information obtained from the imaging device, and so the imaging device and its hardware processor must output both image region information and identifying information as claimed.
8. Regarding claim 3,
Liu discloses the limitations of claim 1 and also:
wherein the hardware processor is configured to set, on the basis of the identification information, a sorting destination on which the object held by the load handling device is to be placed.Lin discloses this action in [0009] and [0078]. Per these paragraphs, the image recognition device 102 can extract barcode data from a package and further can determine destination information for the package as claimed; this computational function is necessarily performed by the hardware processor of the image recognition device.
9. Regarding claim 7,
Liu discloses the limitations of claim 1 and also:
wherein the identification information includes a barcode or a two-dimensional code, and the hardware processor is configured to recognize the identification information.Liu discloses the recognition of barcodes in [0009] and [0018].
10. Regarding claim 8,
Liu discloses:
A sorting system (100: fig. 1) comprising: a load handling device (103: fig. 1);
an imaging device (image recognition device 102: fig. 1) configured to image a region on which an object to be handled by using the load handling device is placed;
and an information processing device (104: fig. 1) configured to control the load handling device,
wherein the information processing device includes: a connection device (105: fig. 1) configured to connect to the imaging device and a hardware processor (component of image recognition device 102: fig. 1) connected to the connection device and configured to acquire image information about the region via the connection device, detect, from the image information, an identification region in which identification information attached to the object is included, and recognize the identification information in the identification region Liu’s image recognition device 102 is a system and not a unitary machine that per [0035]-[0036] comprises devices such as cameras and image sensors as well as a computer with processor and a network interface per [0009] because in addition to merely acquiring images, the image recognition device can also extract barcodes, determine destination information from barcodes, and perform other computational tasks. The image data acquired by a camera or other image sensor part of image recognition device 102 includes information about the region, namely the image itself. Per [0009] the controller obtains barcode information from the imaging device and so the imaging device and its hardware processor must both detect the barcode itself in the image (i.e. detect the identification region) and also recognize identifying (barcode) information as claimed.
11. Regarding claim 9,
Liu discloses:
A non-transitory computer-readable recording medium on which programmed instructions are recorded, the instructions causing a computer (component of image recognition device 102: fig. 1) to execute processing,We consider that Liu’s image recognition device 102 comprises a hardware processor, because a) all networked devices comprise processors and b) image recognition and barcode extraction is a computerized function that requires a processor. It follows that the processor runs programmed instructions derived from a non-transitory computer-readable recording medium because a processor cannot function without a program and because such instructions cannot be persistently stored except on non-transitory media.
the processing comprising: acquiring, from an imaging device (image recognition device 102: fig. 1), image information about a region on which an object to be handled by using a load handling device (103: fig. 1) is placed, the imaging device imaging the region via a connection device (105: fig. 1) which connects to the imaging device;Liu’s image recognition device 102 is a system and not a unitary machine that per [0035]-[0036] comprises devices such as cameras and image sensors as well as a computer with processor and a network interface per [0009] because in addition to merely acquiring images, the image recognition device can also extract barcodes, determine destination information from barcodes, and perform other computational tasks. Any photographic image inherently defines a region of its subject, and the image data acquired by such a device includes information about the region, namely the image itself. Liu’s image recognition device 102 communicates via the network 105 (i.e. via a connection device) as seen in fig. 1.
detecting, from the image information, an identification region in which identification information attached to the object is included; recognizing the identification information in the identification region; and outputting information about the detected identification region and outputting the recognized identification information.According to Liu’s [0036] and [0037] along with fig. 1, the image recognition device 102 is on a network 105 and communicates (outputs) its information to a robotic arm controller 104. Per [0009] and [0078] the controller a) obtains barcode information from the imaging device and b) grasps packages according to information obtained from the imaging device, and so the imaging device and its hardware processor must output both image region information and identifying information as claimed in order for the controller to function as disclosed.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
12. Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Melikian; Simon, US 2015/0197009 (hereinafter Melikian).
13. Regarding claim 2,
Liu discloses the limitations of claim 1 and but not:
wherein the hardware processor is configured to set a holding position at which the load handling device holds the object, the holding position being set within the identification region on the basis of the region information.Lin does not disclose using the identification region (e.g. a barcode region on the object) as a target for a gripper to hold the object.
Melikian, an invention in the field of robotics, teaches:
wherein the hardware processor is configured to set a holding position at which the load handling device holds the object, the holding position being set within the identification region on the basis of the region information.Melikian teaches in [0023] using a barcode or QR code as a target for a robotic manipulator to grasp an object. As the barcode or QR code is within the “identification region” found in the image region, the holding position is set within the identification region as claimed.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to configure the system of Liu, to set a holding position at which the load handling device holds the object, the holding position being set within the identification region on the basis of the region information, as taught by Melikian, because in picking objects from an unordered stack as in the applicant’s fig. 2, an already-recognized barcode region of an object provides a definite, flat target for a gripping device to grip an object, and so no further image analysis would be necessary to determine a target for a robotic manipulator. This teaching results in a savings of computational work and hence real time that would provide an advantageous benefit understood by a person of ordinary skill in the art.
14. Regarding claim 5,
Liu in view of Melikian teaches the limitations of claim 2 and also:
wherein the hardware processor is configured to set the holding position at a substantial surface gravity center or a substantial center of the identification region.Liu discloses the use of a center of a package region in [0013] for gripping purposes. Per the rejection under 35 U.S.C. 112(b) above, we interpret the term “substantial surface gravity center” to be “substantially at the center” in this action due to the indefiniteness of “surface gravity”. Thus Liu discloses the first of the two options of the claim. Because Liu in view of Melikian teaches the use of an identification region as the target for a robotic manipulator to grasp an object, a broad interpretation of “substantial” also admits the second option.
15. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Cai, et al., US 2011/0290880 (hereinafter Cai).
16. Regarding claim 4,
Liu discloses the limitations of claim 1 but not all aspects of:
wherein the hardware processor is configured to calculate a position of the identification region in the region, and the region information includes information indicating the position of the identification region calculated by the hardware processor.Because Liu discloses the acquisition of barcode information from an image in [0009], it could be argued that this process necessarily involves the calculation of the position of the region, but Liu does not disclose this explicitly and so we employ a second teaching reference.
Cai, an invention in the field of barcode decoding, teaches:
wherein the hardware processor is configured to calculate a position of the identification region in the region, and the region information includes information indicating the position of the identification region calculated by the hardware processor.Cai teaches this region-and-position calculation in [0050] based on an image provided as input to the invention.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to configure the system of Liu to calculate a position of the identification region in the region, and the region information includes information indicating the position of the identification region calculated by the hardware processor, as taught by Cai, because in being presented with an image that contains a barcode somewhere in the image it is plainly necessary to determine where the barcode is in order to analyze and extract its information.
17. Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Liu in view of Adachi Takashi, JP 2018171579 (hereinafter Adachi).
Liu discloses the limitations of claim 1 but not:
wherein the identification information includes a character string, the hardware processor is configured to recognize the identification information by OCR processing.Liu does not disclose OCR processing.
Adachi, an invention in the field of sorting automation, teaches the limitation:
wherein the identification information includes a character string, the hardware processor is configured to recognize the identification information by OCR processing.Adachi’s invention is largely about OCR processing, but reading text using OCR from an “identification region” (rectangular portion) of a parcel is taught in [0015].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to configure the system of Liu, wherein the identification information includes a character string, the hardware processor is configured to recognize the identification information by OCR processing, as taught by Adachi because OCR recognition in parcel handling and sorting has been widespread and commonplace for many years and provides a standard alternative to barcode reading where barcodes are either unavailable or are damaged or defective on a particular parcel. As OCR is performed entirely in software, combining Adachi’s teaching would yield routine and predictable results.
Conclusion
18. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The European search report cited CN 109772718 as an alternative X document anticipating some of the claims of the instant application.
19. Any inquiry concerning this communication or earlier communications from the examiner should be directed to LAURENCE RAPHAEL BROTHERS whose telephone number is (703)756-1828. The examiner can normally be reached M-F 0830-1700.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ernesto Suarez can be reached at (571) 270-5565. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERNESTO A SUAREZ/Supervisory Patent Examiner, Art Unit 3655
LAURENCE RAPHAEL BROTHERS
Examiner
Art Unit 3655A
/L.R.B./Examiner, Art Unit 3655