DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed on 05/26/2026 have been fully considered but they are not persuasive.
Applicant argues regarding the newly amended claim language: “Applicant respectfully submits that its claimed invention is patentable over the cited references.”
Examiner notes that the newly amended claim language is addressed in the updated reasons for rejection below.
Applicant argues: “Further, none of cited references a real-time comparison of the three-dimensional localization with the prohibited region. This is because none of the prior art documents is concerned with a detection of a penetration and a non-penetration of a prohibited region. In fact, none of the cited references disclose a prohibited region in a work chamber as claimed. A "prohibited region" is a technical feature due to the comparison with the storing of the coordinates and the comparison with three-dimensional localization. Further, the generation of the signal based on the real-time comparison is also technical feature.”
Examiner notes that the newly amended claim features are addressed in view of a new reference and based on the updated reasons for rejection below.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-7, 10-28 recite the limitation “a computer unit for storing the output of cameras” in independent Claim 1, however Claim 1 has not recited (or cancelled the claim language directed to) cameras as a prior limitation in the claimed arrangement. There is insufficient antecedent basis for this limitation in the claim.
Claim Construction
Note that, for purposes of compact prosecution, multiple reasons for rejection may be provided for a claim or a part of the claim. The rejection reasons are cumulative, and Applicant should review all the stated reasons as guides to improving the claim language and advancing the prosecution toward an allowance.
Claim scope is not limited by claim language that suggests or makes optional but does not require steps to be performed by a method claim, or by claim language that does not limit an apparatus claim to a particular structure. However, examples of claim language, although not exhaustive, that may raise a question as to the limiting effect of the language in a claim are: (A) “adapted to” or “adapted for” clauses; (B) “wherein” clauses; and (C) “whereby” clauses. M.P.E.P. 2111.04. Other examples are where the claim passively indicates that a function is performed or a structure is used without requiring that the function or structure is a limitation on the claim itself. The clause may be given some weight to the extent it provides "meaning and purpose” to the claimed invention but not when “it simply expresses the intended result” of the invention. In Hoffer v. Microsoft Corp., 405 F.3d 1326, 1329, 74 USPQ2d 1481, 1483 (Fed. Cir. 2005). Further, during prosecution, claim language that may or may not be limiting should be considered non-limiting under the standard of the broadest reasonable interpretation. See M.P.E.P. 904.01(a); In re Morris, 127 F.3d 1048, 44 USPQ2d 1023 (Fed. Cir. 1997).
A claim containing a “recitation with respect to the manner in which a claimed apparatus is intended to be employed does not differentiate the claimed apparatus from a prior art apparatus” if the prior art apparatus teaches all the structural limitations of the claim. Ex parte Masham, 2 USPQ2d 1647 (Bd. Pat. App. & Inter. 1987).
Component arrangements or rearrangements which do not modify operation of the device cannot be relied upon to patentably distinguish the claimed invention from the prior art. In re Seid, 161 F.2d 229, 73 USPQ 431 (CCPA 1947); In re Japikse, 181 F.2d 1019, 86 USPQ 70 (CCPA 1950) (shifting the position of the starting switch was not patentable because it would not have modified the operation of the device.
Use of the word “means” (or “step for”) in a claim with functional language creates a rebuttable presumption that the claim element is to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is invoked is rebutted when the function is recited with sufficient structure, material, or acts within the claim itself to entirely perform the recited function.
Absence of the word “means” (or “step for”) in a claim creates a rebuttable presumption that the claim element is not to be treated in accordance with 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph). The presumption that 35 U.S.C. 112(f) (pre-AIA 35 U.S.C. 112, sixth paragraph) is not invoked is rebutted when the claim element recites function but fails to recite sufficiently definite structure, material or acts to perform that function. M.P.E.P. 2181(I), Williamson v. Citrix Online, LLC, 792 F.3d 1339, 1348, 115 USPQ2d 1105, 1111 (Fed. Cir. 2015) (en banc, quoting Watts v. XL Systems, Inc., 232 F.3d 877, 880 (Fed. Cir. 2000); Personalized Media Communications, LLC v. International Trade Commission, 161 F. 3d 696, 704 (Fed. Cir. 1998). A substitute term acts as a generic placeholder for the term "means" and would not be recognized by one of ordinary skill in the art as being sufficiently definite structure for performing the claimed function. "The standard is whether the words of the claim are understood by persons of ordinary skill in the art to have a sufficiently definite meaning as the name for structure." Williamson at 1349; see also Greenberg v. Ethicon Endo-Surgery, Inc., 91 F.3d 1580, 1583 (Fed. Cir. 1996). Specification must disclose adequate structure for each of the claimed functions, and the structure for special purpose functions must be more than simply a general purpose computer or microprocessor, specification must also disclose an algorithm for performing these claimed functions. Williamson at 1351.
Claims 1-7, 10-28 recite generic terms modified by functional language but not modified by structure or a structural term and not naming a structure readily recognized by persons of skill in the art to perform the claimed function. These limitations invoke 35 U.S.C. 112(f) or 35 U.S.C. 112 (pre-AIA ), sixth paragraph, and shall be construed to cover the corresponding structure described in the specification and equivalents thereof.
Claim 1 recites “a tracking system within the work chamber for recording continuous three-dimensional localization of the at least one work glove …” This element recites a generic term (system) modified by functional language, but it is no longer modified by structure. Specification, Page 4, lines 13-14 describes this element as “The tracking system comprises at least a first camera, … and a second camera.”
Claim 1 also recites: “wherein a signal unit is provided, which, based on the real-time comparison, for an operator intervening in the work chamber and/or a monitoring person:” Specification, Page 5 lines 1-18 define the signal unit as not a structure but rather a signal embodying alerts. This element will not be construed as a structural limitation on the claimed apparatus.
This construction applies to Claims 2-7, 10-28 which are dependent on Claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 6-7, 10, 14-18, 21-27 are rejected under 35 U.S.C. 103 as being unpatentable over US 20160350599 to Bataller (“Bataller”) in view of US 20130076898 to Philippe (“Philippe”) in view of US 20160376044 to Procyshyn (“Procyshyn“) also cited in an IDS.
Regarding Claim 1: “An arrangement for monitoring a state and movement sequence in an aseptic work chamber of a containment located in an installation room, comprising: (Note that, a preamble is generally not accorded any patentable weight where it merely recites the purpose of a process or the intended use of a structure, and where the body of the claim does not depend on the preamble for completeness but, instead, the process steps or structural limitations are able to stand alone. See In re Hirao, 535 F.2d 67, 190 USPQ 15 (CCPA 1976) and Kropa v. Robie, 187 F.2d 150, 152, 88 USPQ 478, 481 (CCPA 1951). Also note that arrangements or rearrangements which do not modify operation of the device cannot be relied upon to patentably distinguish the claimed invention from the prior art. In re Seid; In re Japikse. Also see treatment of prior art capability below.)
a tracking system within the [work chamber] (“For example, the system may apply analytics processes to … crowd, track objects across multiple cameras, perform incident detection” Bataller, Paragraph 4. See similarly “a medical product tracking system” in Phillipe, Paragraphs 30, 36.)
at least one [work glove within the maximum gripping range] along x, y and z axes by the tracking system (For example, “the detailed 3D model comprises objects and buildings in the portion of the real-world environment that is shown in the first 2D scene of the first video. In some implementations the 3D model is automatically generated from one or more 2D fields of view of the first camera.” Bataller, Paragraphs 19-20. “apply video analytics to automatically analyze the video information and to generate real-time [continuous] safety information,” Bataller, Paragraph 62. Similarly see “In some embodiments, two cameras (e.g. the cameras 22, 30 or other cameras) may be oriented in the same or a similar direction, which could assist with determining spatial depth [3D] by comparing the images between the two cameras. This may be particularly useful for determining hand signals as described below.” Phillipe, Paragraphs 63, 98. See treatment of user’s hands that use gloves below.)
the tracking system being adapted to record all regions in the work chamber; (Note that this element does not require modification of the tracking system, but rather indicates that it is adapted for recording the desired regions. For example, this application can be embodied in cameras covering the same region from different angles, as in Specification, Page 8 lines 14-25 and Figs. 1A and 1B. Prior art also indicates that the camera coverage of the tracking system can cover all the desired “scenes 100 and 150 depict the real-world environment 102 from differing angles” Bataller, Paragraphs 38, 62, and Figs. 1A and 1B.)
a computer unit for storing the output of cameras; and (This element is interpreted to claim a computer unit that is configured to store. Prior art teaches an example: “The pre-defined 3D model may have been generate automatically based on analysis of previously captured [and stored] video or may have been defined manually by a user and stored.” Bataller, Paragraphs 52, 71, and Video Analytics Platform 210 storing camera videos in Fig. 2. See similarly: “elements may be implemented in computer programs executing on programmable computers which may include at least one processor, at least one data storage device” Philippe, Paragraphs 25, 78.)
at least one three-dimensional or two-dimensional prohibited region, which may be adjoined by a warning region, is defined in the work chamber; … wherein the prohibited region is smaller than the work chamber; (“the 3D model is automatically generated from one or more 2D fields of view … In some cases the defined trip wire comprises … (ii) one more lines defining a closed boundary, drawn in various directions in the 3D model. [0024] In some cases defining a trip wire for performing video analytics comprises identifying user input” Bataller, Paragraphs 20, 23-24. “For example, a trip wire may be used to monitor a particular area” such as “utilization of restricted or forbidden areas” and generate an alarm upon object crossing. See Bataller, Paragraphs 39, 62.)
the coordinates of the prohibited region and optionally of the warning region are stored in the computer unit; and (This element is interpreted to claim a computer unit that is configured to store the coordinates of the prohibited region. Prior art teaches that objects and tripwires are defined by points or edges having 2D and 3D model coordinates that can be compared in Bataller, Paragraphs 39, 23. Further, “The pre-defined 3D model may have been generate automatically based on analysis of previously captured video or may have been defined manually by a user and stored.” Bataller, Paragraph 52.)
wherein a real-time comparison with the prohibited regions and optional warning regions established in the computer unit is carried out in a computer unit; (“A trip wire may be used by a video analytics platform to perform video analytics. For example, a trip wire may be used to monitor a particular area over a given time period … e.g., by comparing 2D coordinates of detected objects against the 2D trip wire” Bataller, Paragraph 39. “video analytics platform 310 may apply video analytics to automatically analyze the video information and to generate real-time safety information,” Bataller, Paragraph 62.)
wherein a signal unit is provided, which, based on the real-time comparison, for an operator intervening in the work chamber and/or a monitoring person: … a) in the event of unallowed penetration into a prohibited region, alerts them acoustically and/or optically and/or by a vibration acting on the body of the operator and/or of the monitoring person; and b) if warning regions are set up, in the event of critical penetration into a warning region as an approach to a prohibited region, informs them acoustically and/or optically and/or by a vibration acting on the body of the operator and/or of the monitoring person.” (First, note that this claim element does not recite a limitation on any particular structure of this claim. A signal unit is provided without limitation that the present claim provides it. Cumulatively, prior art teaches this application: ““For example, a trip wire may be used to monitor a particular area … the video analytics platform may generate an alarm if an object crosses or intersects the trip wire.” Bataller, Paragraphs 39, 62. For example, “video analytics platform 310 may generate a user interface that includes the real-time safety information” Bataller, Paragraph 63. Cumulatively, “For example, a speaker could be used to generate audible alerts” which is a vibration signal. Philippe, Paragraph 60. See statement of motivation below.)
Bataller describes using the tracking system in a wide variety of locations and with respect to a wide variety of objects in Paragraphs 3 and 50. However, Bataller does not the specific examples of locations and objects that are addressed below.
Philippe indicates that such a video tracking system can be applies to monitory the type of location and the type of objects described in the claim language below:
a work chamber; (For example, the chamber can be a storage compartment, the storage depot, or the room in front of the storage depot as in Philippe, Paragraphs 30 and 92. Cumulativley, “In one example, the chamber 20 includes one or more glove holes” Procyshyn, Paragraph 25. See statement of motivation below.)
the work chamber having at least one work glove which projects into the work chamber, for extension in the work chamber as far as a maximum gripping range in the three spatial axes; (“A non-exhaustive list of exemplary medical products could include … gloves” that can be worn and used within any portion of the work chamber within reach. Philippe, Paragraph 27. Cumulatively, the object can be a hand wearing a work glove where “The movement and/or shape of the hand Bl can then be tracked” wherever the glove extends within view of the cameras as in Phlippe Paragraphs 62, 95 and similarly in Procyshyn, Paragraph 25. See detailed treatment of this embodiment below.)
[a tracking system] within the work chamber (“For example, with reference to the screenshot 200 shown in FIG. 7, the imaging unit 20 may observe an image 201 of the room in front of the storage depot 12” in this case the room can embody the working chamber and the storage compartments embody the inspection areas. See Phillipe, Paragraphs 92, 98.)
[including a plurality of cameras for recording continuous three-dimensional localization] of the at least one work [glove] within the maximum gripping range along x, y and z axes by the tracking system (“In some embodiments, two cameras (e.g. the cameras 22, 30 or other cameras) may be oriented in the same or a similar direction, which could assist with determining spatial depth [3D] by comparing the images between the two cameras. This may be particularly useful for determining hand signals as described below.” Phillipe, Paragraphs 63, 98. Thus, cameras capture 3D for localization of user’s hands wherever they reach within the room, and this computer vision process is substantively identical to tracking the user’s hands wearing gloves. See treatment of user’s hands that use gloves below.)
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to use a video monitoring and computer vision tracking system that track objects in Bataller to perform video tracking and computer vision in a clean hospital setting of Philipe, in order to locate and monitor medical-type objects of interest in a clean environment such as a storage room in a healthcare facility. See, Phillipe, Paragraphs 2-3.
Bataller and Phillipe describe video tracking systems that are generally applicable to many locations and many types of objects to be tracked including objects in work chambers, however they do not provide examples of rooms having the particular designation or containing the particular objects in the claim language below.
Procyshyn teaches using optical monitoring in the context of a substantively similar room and equipment:
arrangement for monitoring … in an aseptic work chamber of a containment located in an installation room, comprising: (First, as noted above, a recitation with respect to the manner in which a claimed apparatus is intended to be employed or arranged does not differentiate the claimed apparatus from a prior art apparatus where the prior art apparatus teaches all the structural limitations of the claim.
Cumulatively, under the broadest reasonable interpretation consistent with the specification and ordinary skill in the art, an aseptic work chamber can be a clean room that is cleaned or sterilized. See Specification, Page 5, lines 17-18. The cleanroom in Procyshyn is similarly cleaned and sterilized: “Articulated cleanroom robots have been employed which utilize internal negative pressure with an exhaust to generate cleanroom capability. With the chemical sterilization and handling of potent drugs within the isolator [chamber]” Procyshyn, Paragraph 4. Thus, the camera tracking system as in the prior art is known to be employed in many varieties of locations including locations that are similar in function and purpose to the intended location of the claims.)
at least one work glove [which projects into the work chamber, wherein the respective work glove can be extended in the work chamber as far as a maximum gripping range in the three spatial axes, and wherein:] (“In one example, the chamber 20 includes one or more glove holes 21 disposed in the walls of the chamber 20. The glove holes 21 can be used to manually manipulate objects within the chamber 20 without opening the chamber 20 or otherwise compromising the environmental conditions within the chamber 20.” Procyshyn, Paragraph 25. It is understood that such a work glove naturally has a maximum gripping range within the chamber such that it operates “without opening the chamber 20 or otherwise compromising the environmental conditions within the chamber.”)
Finally, Procyshyn also provides the context of optical monitoring: “the sensor 12 is an optical sensor, a camera system, or a laser system. The sensor 12, in one example, is mounted at a top surface of the chamber 20 and is positioned to sense an area within the chamber 20, as portrayed by a sensing cone 14. For instance, the optical sensor 12 can be configured to locate containers 90 within the sensing cone 14 and target centers of the openings of the containers 90.” Procyshyn, Paragraph 27.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to use an optical monitoring and computer vision tracking system that track objects in a clean hospital setting of Philipe to perform optical tracking and computer vision in a containment room such as cleanroom with chambers accessible by a glove, as taught in Procyshyn, in order to locate and monitor objects of interest in a medication filling environment of Procyshyn as well as in a medication storage environment of Philipe. See, Procyshyn, Paragraph 27.
Finally, in reviewing the present application, there does not seem to be objective evidence that the claim limitations are particularly directed to: addressing a particular problem which was recognized but unsolved in the art, producing unexpected results at the level of the ordinary skill in the art, or any other objective indicators of non-obviousness.
Regarding Claim 2: “The arrangement as claimed in claim 1, characterized in that wherein:
a) the at least one work glove comprises two work gloves and each work glove may respectively hold an object; and (Prior art provides for multiple “glove holes 21 can be used to manually manipulate objects within the chamber 20 without opening the chamber 20 or otherwise compromising the environmental conditions within the chamber 20.” See, Procyshyn, Paragraph 25, Fig. 1, and statement of motivation in Claim 1.)
b) recordings of the tracking system are used for continuous three-dimensional localization both work gloves.” (Note that this claim is directed to a physical arrangement of components, it is not a method and does not affirmatively perform a step of “performing three-dimensional localization.” To the extent this claim element recites an intended use of the arrangement of objects in a room for the purposes of localizing the objects based on a visual recording, Phillippe indicates that the tracking system locates the objects (such as gloves) in the room by their 3D location in the room and with respect to other objects in the room in Paragraphs 27-32. Therefore, this is a known use of a tracking system in a room of objects.)
Regarding Claim 6: “The arrangement as claimed in claim 1, wherein the tracking system comprises: … a) a first camera, which has a first acquisition angle; and … b) a second camera, which has a second acquisition angle; wherein: … ca) the first camera monitors and records two spatial axes and the second camera monitors and records the third spatial axis; … cb) the first acquisition angle and the second acquisition angle overlap so that all regions in the work chamber are monitored and recorded; and … cc) the recorded states and movement sequences are stored as 3D data in the computer unit.” (“In some embodiments, two cameras (e.g. the cameras 22, 30 or other cameras) may be oriented in the same or a similar direction, which could assist with determining spatial depth by comparing the images between the two cameras.” Thus the first camera captures 2D and the addition of the second camera enables capturing 3D. See Phillipe, Paragraph 63 and imaging the entire compartment in Fig. 16. Cumulatively, “In such embodiments the imaging units 20 may send raw image data 112 (e.g. a raw video stream) to an image processing server 110” See Phillipe, Paragraph 84.)
Regarding Claim 7: “The arrangement as claimed in claim 6, further compnsing
a third camera with a third acquisition angle wherein: (“Each imaging unit includes one or more cameras adapted to visually inspect storage compartments e.g. drawers in a cabinet, bins on open shelves, etc.) and determine the presence or absence of medical products therein … one or more of the imaging units 20 may communicate with the server 102” where more cameras may be naturally required for larger compartments, and more imaging units may be required for larger number of compartments. See Phillipe, Paragraphs 30, 77.)
a) the first camera with its first acquisition angle monitors and records the first spatial axis, the second camera with its second acquisition angle monitors and records the second spatial axis and the third camera with its third acquisition angle monitors and records the third spatial axis; … b) the acquisition angles overlap so that all regions in the work chamber are recorded; and … c) the recorded states and movement sequences are stored as 3D data in the computer unit.” (As noted in Claim 6: “In some embodiments, two cameras (e.g. the cameras 22, 30 or other cameras) may be oriented in the same or a similar direction, which could assist with determining spatial depth by comparing the images between the two cameras.” Thus, the first camera captures 2D and the addition of the second camera enables capturing 3D. See Phillipe, Paragraph 63 and imaging the entire compartment in Fig. 16. Cumulatively, “In such embodiments the imaging units 20 may send raw image data 112 (e.g. a raw video stream) to an image processing server 110” See Phillipe, Paragraph 84. See addition of further cameras with further fields of view above.)
Regarding Claim 10: “The arrangement as claimed in claim 9, characterized in that wherein the tracking system further comprises biometric detection apparatus to detect the biometric features of an operator positioned at the work chamber.” (“For example, at least one of the first and second cameras 22, 30 may be used to capture facial features [biometric features] and/or gestures (such as facial gestures or hand signals) to communicate additional information to the imaging unit 20. This may be particularly useful for associating patient information with particular medical products, which can be important for accurate patient charging, for facial recognition and/or for access control.” Phillipe, Paragraph 87.)
Regarding Claim 14: “The arrangement of claim 1, wherein the prohibited region comprises a three-dimensional region.” (For example, the 3D monitored region (designated for some objects in the video and prohibited to others) can be a storage compartment. See Phillippe, Paragraphs 62, 69-71.)
Regarding Claim 15: “The arrangement of claim 1 wherein the prohibited region comprises a two-dimensional region.” (For example, the 2D monitored region (designated for some objects in the video and prohibited to others) can be a designated storage region in a particular storage compartment, for example: “the various storage regions 40a, 40b, 42a, 42b in a drawer 14a can be defined by one or more dividers 44, 46.” See Phillippe, Paragraphs 66, 69-71, and Fig. 3.)
Regarding Claim 16: “The arrangement of claim 1 wherein a warning region adjoins the prohibited region.” (For example, regions designated for certain objects can border regions prohibited for those objects: “the various storage regions 40a, 40b, 42a, 42b in a drawer 14a can be defined by one or more dividers 44, 46. … a first type of medical product 52 ( e.g. a syringe) is located within each of the storage regions 40a, 40b, a second type of medical product 54 ( e.g. a box of needles) is located in the secondary storage region 42b,” See Phillippe, Paragraphs 66, 70, and Fig. 3.)
Claim 17 is rejected for reasons stated for Claim 16 in view of the Claim 14 rejection.
Claim 18 is rejected for reasons stated for Claim 16 in view of the Claim 15 rejection.
Regarding 21: “The arrangement of claim 16 wherein the signal unit alerts of unallowed penetration into a warning region.” (For example, a speaker could be used to generate audible alerts ( e.g. beeps) or express a recorded voice,” in response to the wrong object being present in the wrong region or in the wrong part of the monitored [designated] region. Phillipe, Paragraphs 60, 69-72. Note that while Philippe does not name its regions prohibited or warning, it does designate regions by number and by content of intended / unintended objects. See, Phillipe, Paragraphs 66-72.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to issue an alert of a prohibited event, (such as an unallowed penetration of a particular object into a particular region in the room) when such an event is detected.)
Regarding Claim 22: “The arrangement of claim 1 wherein the signal unit alerts with an acoustic signal.” (“For example, a speaker could be used to generate audible alerts” Philippe, Paragraph 60.)
Regarding Claim 23: “The arrangement of claim 1 wherein the signal unit alerts with a vibration signal.” (“For example, a speaker could be used to generate audible alerts” which is a vibration signal. Philippe, Paragraph 60.)
Regarding Claim 24: “The arrangement of claim 1, wherein individual surface sections are a prohibited region.” (Note that objects can be within defined detection zones (such as a prohibited region) that can be within defined regions (such as a hazard region); “During use the imaging unit 20 can define one or more detection zones 56, 58 around one or more of the medical products 52 ( e.g. the detection zone 56 is around the syringe in storage region 40a) and/or within one or more regions 40a, 40b, 42a, 42b ( e.g. the detection zone 58 is around storage region 42a). The detection zones 56, 58 allow the imaging unit 20 to determine whether there is an object in the corresponding region 40a, 40b, 42a, 42b.” See Phillipe, Paragraph 71. See obviousness of naming regions in Claim 1.)
Regarding Claim 25: “The arrangement of claim 24, wherein the coordinates of the prohibited region are stored in a computer unit.” (“In some embodiments, the imaging unit 20 may adapted to learn the particular layout of each particular storage compartment 14 (e.g. the size and shape of the regions 40a, 40b, 42a, 42b may be observed and stored in a database” Phillipe, Paragraph 68.)
Regarding Claim 26: “The arrangement of claim 1, wherein the entire bottom is a hazard.” (Note that objects can be within defined detection zones around one or more products (such as a prohibited region) that can be within defined regions (such as a hazard region); “During use the imaging unit 20 can define one or more detection zones 56, 58 around one or more of the medical products 52 ( e.g. the detection zone 56 is around the syringe in storage region 40a) and/or within one or more regions 40a, 40b, 42a, 42b ( e.g. the detection zone 58 is around storage region 42a). The detection zones 56, 58 allow the imaging unit 20 to determine whether there is an object in the corresponding region 40a, 40b, 42a, 42b.” See Phillipe, Paragraph 71. See obviousness of naming regions in Claim 1.)
Regarding Claim 27: “The arrangement of claim 26, wherein coordinates of the hazard are stored in a computer unit.” (“It will be appreciated that the detection region 24 may vary in size and shape, particularly to accommodate different storage compartments 14. … In some embodiments, the imaging unit 20 may adapted to learn the particular layout of each particular storage compartment 14 (e.g. the size and shape of the regions 40a, 40b, 42a, 42b may be observed and stored in a database” Phillipe, Paragraphs 47, 68.)
Claims 3-5, 12-13, 19-20, 28 are rejected under 35 U.S.C. 103 as being unpatentable over US 20160350599 to Bataller (“Bataller”) in view of US 20130076898 to Philippe (“Philippe”) in view of US 20160376044 to Procyshyn (“Procyshyn“) also cited in an IDS, and in view of US 20200368616 to Delamont (“Delamont”).
Regarding Claim 3: “The arrangement as claimed in at least one of claims 1 and 2, characterized in that claim 2, wherein:
a) the spatial positions of work gloves and objects within the xyz coordinate system formed by the three spatial axes are recorded with [a timestamp] in the computer unit; and (“In some embodiments, two cameras (e.g. the cameras 22, 30 or other cameras) may be oriented in the same or a similar direction, which could assist with determining spatial depth by comparing the images between the two cameras. This may be particularly useful for determining hand signals as described below.” Phillipe, Paragraph 63. Thus cameras capture 3D for localization of user’s hands. See treatment of user’s hands that use gloves and statement of motivation in Claim 1.)
b) a real-time comparison with the prohibited regions is established in the computer unit.” (See Bataller, Paragraphs 39 and 62. Similarly, “The detection zones 56, 58 allow the imaging unit 20 to determine [comparison] whether there is an object in the corresponding region 40a, 40b, 42a, 42b.” Phillipe, Paragraph 71. “For example, if the user A removes a particular medical product, this information can be sent to the server 102 (in some embodiments in real time or substantially real time, … The image processing server 110 can then analyze the raw image data 112” Phillipe, Paragraphs 82, 84. Thus, the regions can be designated for certain objects and prohibited for others. See statement of motivation in Claim 1.)
Bataller, Phillipe and Procyshyn do not teach: “recorded with a timestamp” however, video is commonly recorded using timestamps in order to track playback and join different streams of video:
Delamont teaches the above claim feature in the context of tracking user movements and synchronizing captured, tracked, and virtual images: “allows adjustments to be made in real-time in the 3D projection mapping of augmented images over the real-world AI Character 94. Here the game server 88 or host 89 may know in advance of the move and be provided with timestamps for the coordinates, where movements are a result of a rag doll simulation for example, in which in precise time sequence with the movements of the AI Characters 94 body adjustments may be made” Delamont, Paragraph 1065.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to supplement the teachings of P Phillipe and Procyshyn to provide video with timestamps as taught in Delamont, in order to provide a “precise time sequence with the movements” that are being tracked. See, Delamont, Paragraph 1065.
Finally, in reviewing the present application, there does not seem to be objective evidence that the claim limitations are particularly directed to: addressing a particular problem which was recognized but unsolved in the art, producing unexpected results at the level of the ordinary skill in the art, or any other objective indicators of non-obviousness.
Regarding Claim 4: “The arrangement as claimed in claim 1, wherein:
a) containers are temporarily located in the work chamber; and (“recognizing and properly interpreting user activities, such as a user approaching a storage compartment, opening a drawer, removing a medical product from the storage compartment, putting an item in the storage compartment (e.g. restocking a medical product), or changing the position of a medical product” Phillipe, Paragraph 39. See similarly in Procyshyn, Paragraph 7 and statement of motivation in Claim 1.)
b) the spatial positions of the containers are recorded in three dimensions [with a timestamp] in the computer unit.” (“The detection zones 56, 58 allow the imaging unit 20 to determine whether there is an object in the corresponding region 40a, 40b, 42a, 42b.” Phillipe, Paragraph 71. “A non-exhaustive list of exemplary medical products could include medications, intravenous solutions, catheters, tubes, implants, pacemakers, gloves, needles, syringes, and so on,” which include examples of containers. Phillipe, Paragraph 27. See recording of 3D video with determination of spatial depth and statement of motivation in Claim 1.)
“recorded with a timestamp” (Delamont teaches the above claim feature in the context of tracking user movements and synchronizing captured, tracked, and virtual images: “allows adjustments to be made in real-time in the 3D projection mapping of augmented images over the real-world AI Character 94. Here the game server 88 or host 89 may know in advance of the move and be provided with timestamps for the coordinates, where movements are a result of a rag doll simulation for example, in which in precise time sequence with the movements of the AI Characters 94 body adjustments may be made” Delamont, Paragraph 1065. See statement of motivation in Claim 3.)
Regarding Claim 5: “The arrangement as claimed in claim 1, further comprising [virtual reality glasses] a) to display the prohibited regions and optional warning regions defined in the computer unit as graphics elements; and b) to display a surface to be cleaned according to the cleaning and disinfection program saved in the computer unit.” (Phillipe uses a display to guide the user to proper zones (and thus away from improper zones):“In some embodiments, the imagining unit 20 may inform the user of the proper storage compartment in other ways, which may be audible (e.g. a wired or wireless speaker), visual ( e.g. using a display such as an LCD display,” for example: “medical products from the corresponding secondary storage region 42b [prohibited or warning regions] can be rotated or moved into the primary storage region 42a.” Phillipe, Paragraphs 103-104. See statement of motivation in Claim 1.)
However, Bataller, Phillipe and Procyshyn do not teach to display using: “virtual reality glasses.”
Delamont teaches the above claim feature in the context of tracking user movements and synchronizing captured, tracked, and virtual images: “Alternatively, virtual retinal display (VRD) may be used. For example, low cost alternative liquid crystal display (LCD) shutter 3D glasses or wearable display could be used to display augmented scenes” Delamont, Paragraph 419.) See statement of motivation in Claim 3.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to supplement the teachings of Phillipe and Procyshyn to use virtual reality glasses as taught in Delamont, “for presenting images that create the illusion of an image having three dimensional form.” Delamont, Paragraph 420.
Finally, in reviewing the present application, there does not seem to be objective evidence that the claim limitations are particularly directed to: addressing a particular problem which was recognized but unsolved in the art, producing unexpected results at the level of the ordinary skill in the art, or any other objective indicators of non-obviousness.
Regarding Claim 12: “The arrangement as claimed in at least one of claims l to 11, characterized in that claim 1, wherein:
a) the at least one work glove of the work chamber includes systematically distributed marking points; and (“the real-world 3D mesh is formed of multiple surface display panel faces 57L, 57R, 57 overlaid on the physical IR Laser Gun Apparatus 47 device and real-world game object in which the object/device is formed of real world vertices/points, edges, faces, polygons and surfaces [another words marking points] in three-dimensional space of the real-world” Delmont, Paragraphs 568, 574. See statement of motivation in Claim 5.)
b) the volume body formed by the work glove is recorded as a CAD wire model by the tracking system.” (“Other usages of the devices cameras 50 include the generating of 3D mesh data, wireframes and 3D models containing geometric, depth and volumetric data of the real-world objects, surfaces and surrounds using spatial mapping techniques, which is used in the display of augmented images via the users Augmented Reality ("AR") display” Delmont, Paragraph 513. See statement of motivation in Claim 5.)
Claim 13 is rejected for reasons stated for Claim 12 in view of the Claim 2 rejection.
Regarding Claim 19: “The arrangement of claim 6 wherein the 3D data comprises video streams.” (Phillippe teaches generating stereoscopic 3D but does not discuss streaming this information. See Philippe, Paragraph 84.
Delamont teaches “data stream sources of differing 2D stereoscopic images [3D data] or video to be displayed in front of each of the user's eye's respectively to achieve the display of a virtual image which appears to the user as a 3D formed image” Delamont, Paragraph 1065. See statement of motivation in Claim 3.)
Claim 20 is rejected for reasons stated for Claim 19 in view of the Claim 7 rejection.
Regarding Claim 28: “The arrangement according to claim 1, wherein the step of storing the output of the cameras comprises a recording with a timestamp.” (“Here the game server 88 or host 89 may know in advance of the move and be provided with timestamps for the coordinates, where movements are a result of a rag doll simulation for example, in which in precise time sequence with the movements of the AI Characters 94 body adjustments may be made” Delamont, Paragraph 1065. See statement of motivation in Claim 3.)
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over US 20160350599 to Bataller (“Bataller”) in view of US 20130076898 to Philippe (“Philippe”) in view of US 20160376044 to Procyshyn (“Procyshyn“) also cited in an IDS, and in view of US 20200368616 to Delamont (“Delamont”) and further in view of US 20150077528 to Awdeh (“Awdeh”).
Regarding Claim 11: “The arrangement as claimed in claim 5, characterized in that wherein the virtual reality glasses display,
a prescribed intensity for the surface cleaning,
As noted in Claim 5, Phillipe uses a display to guide the user to perform a desired action such as rearranging, ordering, and orienting stored objects, which is a form of cleaning. Phillipe, Paragraphs 103-104.
However, Philippe, Procyshyn, and Delamont are not concerned with cleaning surfaces at pressure, and so do not provide guidance to these actions specifically.
Awdeh teaches the above claim feature (cleaning surfaces at specified intensities) in the context of “medical visualization systems” similar to visual systems in Philippe, Procyshyn: “Although not shown, one skilled in the art will also appreciate that system 10 may be adopted to assist the surgeon to perform various other procedures known in the art. For example, after the sculpt procedure, eyewear device 7 may display any one or more of the following messages: … [first intensity] a text message "QUADRANT REMOVAL" prompting the surgeon to remove large chunks of the cataract using the vacuum instrument; … [second intensity] a text message "EPINUCLEUS" prompting the surgeon to remove remaining epinucleus layer of the lens; … [third intensity] a text message "IRRIGATION & ASPIRATION" prompting the surgeon to clean and remove any leftover material more thoroughly using smaller instruments with lower vacuum pressure; … [fourth intensity] a text message "POLISH" prompting the surgeon to posh the lens capsule;” Awdeh, Paragraph 85.
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to supplement the teachings of Philippe, Procyshyn, and Delamont to clean surfaces at a prescribed intensity as taught in Awdeh, so “that system 10 may be adopted to assist the surgeon to perform various other procedures known in the art.” Awdeh, Paragraph 85.
Finally, in reviewing the present application, there does not seem to be objective evidence that the claim limitations are particularly directed to: addressing a particular problem which was recognized but unsolved in the art, producing unexpected results at the level of the ordinary skill in the art, or any other objective indicators of non-obviousness.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20190362562 to Benson (“Benson”) is relevant to the application of an augmented reality display, such as using 3D goggles, to a system that captures and processes 3D video information using multiple cameras.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MIKHAIL ITSKOVICH whose telephone number is (571)270-7940. The examiner can normally be reached Mon. - Thu. 9am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Joseph Ustaris can be reached at (571)272-7383. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MIKHAIL ITSKOVICH/Primary Examiner, Art Unit 2483
Prosecution Insights