DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
Claims 1-6 have been interpreted under 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) to not invoke 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) claim interpretation. Since the claimed “work assist device configured to” in claim 1 is considered to have sufficient structure and considered to not be a generic placeholder then items comprising “work assist device” inherit this claim interpretation position.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 5, and 6 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Oliveira et al., WO 2018166652 A1, hereinafter, Oliveira.
Claim 1:
1. A work facility comprising:
a container arrangement section in which a plurality of containers are arrangeable (Oliveira: Figs. 2 and 7-10, page 5 line 28 to page 6 line 27 and page 9 line 13 to page 10 line 34.);
a work area for a worker to perform placement and removal work of placing and removing articles into and out of the containers, the work area located adjacent to the container arrangement section (Oliveira: Figs. 2 and 7-10, page 5 line 28 to page 6 line 27 and page 9 line 13 to page 10 line 34.); and
a work assist device configured to assist the worker in the placement and removal work (Oliveira: augmented reality system, page 6 lines 6-27.), wherein the work assist device comprises:
a display device that the worker is capable of wearing in front of the eyes of the worker and that is configured to show the worker an augmented reality image (Oliveira: display device, page 3 lines 10 to16 and page 3 line 30 to page 4 line 9, page 3 line 16 “display data presentable to the user via a display device, such as a head on display”, page 4 lines 1-4 “The display device might be a head up display device, which is worn by the user like glasses, and which preferably allows the user to see the reality/the environment and additional information. Preferably, the reality is seen through the glasses and additional information is projected on the glasses.”.);
an article information acquisition system configured to acquire article information that is information regarding the articles contained in the containers disposed in the container arrangement section (Oliveira: identification process, page 6 line 6 to page 7 line 15, page 9 lines 16-27 “The rack system 86 has one or more opened sides, containing physical or virtual markers 78 that allow detecting the presence and/or position of the racks and its contents or containers in a working zone 106. Physical or virtual markers 78 might be present or identified to allow the identification of compartments, shelves within the racks or containers. Object identifiers 80 could be present to uniquely identify the container and its goods. The identification of items will be done using its shape and/or an object identifier (e.g. barcode, RFID tag, etc.). The data captured by at least one sensing device may be transmitted to the processing device, where the marker, location, position or object can be identified by the processing device. The processing device can generate a decision on the identification of a position, location or product and sent the output to the augmented device so it can identify and generate and augmented display image of the device highlighting the identified object.”, page 10 lines 4-13 “Additionally, physical or virtual markers can be present or identified to allow the identification of compartments 88, shelves within the racks or containers. An object identifier will be present to uniquely identify the container and its goods. The identification of goods will be done using its shape and/or an object identifier (e.g. barcode, RFID tag, etc.). The data captured by at least one sensing device may be transmitted to the processing device, where the marker, location, position or object can be identified by the processing device. The processing device can generate a decision on the identification of a position, location or product and sent the output to the augmented device so it can identify and generate and augmented display image of the device highlighting the identified object”.); and
a control system configured to control the display device, and wherein the control system generates a container transmission image that is the augmented reality image making at least one container out of the containers disposed in the container arrangement section appear transparent and showing the worker an appearance of the articles contained in the at least one container, based on the article information acquired by the article information acquisition system, and causes the display device to display the container transmission image (Oliveira: augmented reality system’s processing device generates image for display on the display based on the article information acquired by the article information acquisition system,
Fig. 2 illustrate an augmented reality container transmission image 24 displayed over pick or put away container, refer to page 5 line 20 to page 8 line 16,
page 2 line 29 to page 3 line 2 “Preferably, the displayed information associated to the at least one item includes highlighting an area of any transport unit or of any shelf, compartment, and boxed area. Transport units might be a rack, a tote, a bin, a box, and/or a pallet. Thus, visual guidance can be provided informing the user from where items need to picked from and/or put away to by using visual clues, e.g. arrows, pointers, highlighted areas), wherein the user can be guided by the augmented reality system through the complete process from pick to put away.”,
page 6 lines 11-13 “When a user needs to pick or put away an item 22 from or to these areas they are marked by highlighting means 24, for example by using a different colour or shape.”, highlighting with color or shape covers the BRI of the claimed “a container transmission image that is the augmented reality image making at least one container out of the containers disposed in the container arrangement section appear transparent and showing the worker an appearance of the articles contained in the at least one container”.).
Claim 5:
5. The work facility according to claim 1,
wherein the container arrangement section is a work rack wherein a plurality of shelves are arranged in an up-down direction and the containers are disposed on the shelves (Oliveira: Figs. 2 and 7-10.), and
wherein the control system causes the display device to display the container transmission image of the containers disposed at a height higher than or equal to a preset height (Oliveira: Fig. 2 illustrates transmission image 24 displayed over a “pick or put away” container 20, page 6 lines 9-13 “The augmented reality system is able to enhance the reality by creating virtual shelves or containers 20 that represent the defined areas of a physical rack, here rack 14. When a user needs to pick or put away an item 22 from or to these areas they are marked by highlighting means 24, for example by using a different colour or shape.”.).
Claim 6:
6. The work facility according to claim 1, wherein the article information acquisition system further comprises a work instruction information acquisition unit configured to acquire work instruction information that is information indicating content of an instruction for the worker regarding the placement and removal work, and wherein the control system generates a work instruction image that is an image indicating the work instruction information, and causes the display device to display the work instruction image in a mode different from the container transmission image (Oliveira:
identification process, page 6 line 6 to page 7 line 15 and page 8 lines 9-16, acquires work instruction information “pick or packaging instructions” 26, 56 and augmented reality system’s processing device generates image 26, 56 for display and displays image 26, 56 on the display.).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-4 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The claimed “the article information comprises information indicating a shape and an arrangement of the articles, the shape and the arrangement being results of the image recognition processing” is indefinite because this claim limitation is unclear if the shape and arrangement of the articles is limited to articles within the container.
Dependent claims 3 and 4 inherit and do not correct this indefinite issue.
Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yamashita, EP 4134337 A1, describes augmented reality with regard to picking station, refer to paragraph [0041], and describes identifying items to be picked, refer to paragraph [0024].
Leisieur et al., US Patent Application Publication No. 2019/0043004, describes augmented reality with regard to picking and loading packages including projecting a virtual package at the planned package location, making real-time updates/adjustments to the virtual package being projected, such that the shape and perspective of the virtual package projected are updated based on the location of the associate, and as the associate places the package at the planned package location, the augmented reality device can project a confirmation that the package is correctly placed.
Allowable Subject Matter
Claims 2-4 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Claim 2 compared with Oliveira follows:
2. The work facility according to claim 1, wherein:
each of the containers is open on an upper side thereof (Oliveira: FIGs. 6a-6c.), the article information acquisition system comprises:
a camera configured to capture an image of inside of the containers from above the containers (Oliveira: FIGs. 6a-6c and page 3 lines 10-16 “Preferably, the displayed information associated to the at least one item includes information whether the picking process is completed. This includes order information, pick information, task information that allows the user to understand if an order for a customer has been completed, if a task part of an order pick has been completed, and/or information about the current pick task at hand. The augmented reality system used in the method could comprise a processing device which is capable to receive the required input data, such as sensing data of the work zone, e.g. from one or more sensing device(s) such as camera(s), laser scanner(s), etc., process the data, e.g. by means of a processor or computer and preferably using at least one of a memory, a central processing unit (CPU), a communication connection to an order fulfillment system, a database, an interface to a display device and generate display data presentable to the user via a display device, such as a head on display.”.); and
an image recognition processing unit configured to perform image recognition processing on the image captured by the camera (Oliveira: identification, page 6 line 34 to page 7 line 5 “The identification of the items 22 will be done using the item's shape and/or an object identifier (e.g. barcode, RFID tag, etc.) coupled to or attached to corresponding item 22. The data captured from the work zone by at least one sensing device (e.g. one or more cameras, particularly stereoscopic cameras) may be transmitted to the processing device, which is capable to perform at least one of the steps: identify item 22, determine location and/or orientation of item 22.”, page 9 line 21-24 “The identification of items will be done using its shape and/or an object identifier (e.g. barcode, RFID tag, etc.). The data captured by at least one sensing device may be transmitted to the processing device, where the marker, location, position or object can be identified by the processing device.”, page 9 lines 21-27 “The identification of goods will be done using its shape and/or an object identifier (e.g. barcode, RFID tag, etc.). The data captured by at least one sensing device may be transmitted to the processing device, where the marker, location, position or object can be identified by the processing device. The processing device can generate a decision on the identification of a position, location or product and sent the output to the augmented device so it can identify and generate and augmented display image of the device highlighting the identified object.”, page 10 lines 26-31 “The identification of goods will be done using its shape and/or an object identifier (e.g. barcode, RFID tag, etc.). The data captured by at least one sensing device may be transmitted to the processing device, where the marker, location, position or object can be identified by the processing device. The processing device can generate a decision on the identification of a position, location or product.”.), and
the article information comprises information indicating a shape and an arrangement of the articles, the shape and the arrangement being results of the image recognition processing (Oliveira: article information comprising product images 28,48 covers shape, however, while Fig. 6c illustrates image inside container having physical items 22 arranged in an arrangement Fig. 6c fails to teach or suggest “the article information comprises information indicating a shape and an arrangement of the articles, the shape and the arrangement being results of the image recognition processing”.).
The prior art of record fails to teach or suggest in the context of independent claim 1 and dependent claim 2 “and the article information comprises information indicating a shape and an arrangement of the articles, the shape and the arrangement being results of the image recognition processing”.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEFFERY A BRIER whose telephone number is (571)272-7656. The examiner can normally be reached on Mon-Fri from 8:30am-3:00pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu, can be reached at telephone number 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
JEFFERY A. BRIER
Primary Examiner
Art Unit 2613
/JEFFERY A BRIER/Primary Examiner, Art Unit 2613
Prosecution Insights