DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation is “mixing unit” in claims 5-6, 19-20.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-6, 8-10, 13-14, 16, 19-20 rejected under 35 U.S.C. 103 as being unpatentable over Geric [US 20230051869 A1] in view of Guha [US 20200396379 A1].
As per claim 1, Geric teaches an augmentation overlay device (Geric Fig 7), comprising:
an image source configured for providing live image data of a region of interest in a form of a live image pixel stream (Geric Fig 7, ¶0042 “Camera images from instrument 53 are coupled to a controller (e.g., image processor) 57” ¶0054-¶0055 CCD camera /visible pixel, ¶0051 “a real, live view of the scene around the wearer …”);
a device configured for providing overlay data (Geric ¶0052 “a thermal camera coupled with the harvesting instrument (e.g., carried along with the endoscope) so that the image data includes a temperature overlay based on thermograms captured by the thermal camera”);
a composer device (Geric Fig 7 Augmented-reality display 60),
wherein the composer device is a hardware device and a function of the hardware device is implemented in hardware (Geric ¶0045),
wherein the composer device is configured for:
providing display data in the form of the live image pixel stream by overlaying the provided live image data with the provided overlay data (Augmented-reality display 60 is configured to render glyphs (e.g., text, symbols, colored overlays, etc.) …” ¶0059 “an AR display which includes a video overlay 91 (streaming a live endoscopic view from a visible-light camera).”)
determining a restricted area in the provided live image data (Geric ¶0052 “Restriction of temperature overlay 70 to just the significant locations can be obtained by 1) using pattern recognition … or 2) by only representing temperatures”),
wherein the provided overlay data comprises restricted area overlay data that is associated with the restricted area of the live image data and free overlay data that is associated with an area of the live image data outside of the restricted area, and restricting overlay via the restricted area overlay data (Geric Fig 9, Geric ¶0052 “the temperature overlay may be limited to relevant surfaces within the visible-light view such as the side branches, target vessel, and tunnel wall. Restriction of temperature overlay 70 to just the significant locations…” the relevant surface to where overlay is restricted corresponds to the claimed area of the live image data outside of the restricted area, and the other surfaces are restricted areas. The data corresponding to these regions are the free overlay data and the restricted area overlay data respectively); and
a display device configured for receiving and displaying the provided display data (Geric Fig 8 view field 61).
The only difference from the claim is that Geric does not teach receiving overlay (thermal data here), from a data server.
Guha, in a related field of capturing and overlaying data teaches receiving overlay (thermal data here), from a data server (Guha ¶0030 “the server-side sensor analytics system 134 may be configured to implement additional sensor data analytics functionality not implemented by the sensor analytics system 114. For example, the server-side sensor analytics system 134 may generate trend data that captures thermal conditions of the one or more objects of interest over time”).
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the apparatus in Geric, by utilizing data from server for providing additional sensor analytics functionality like thermal trend data to be generated and displayed (Guha ¶0030-¶0031)
As per claim 5, Geric in view of Guha further teaches wherein the composer device comprises a mixing unit that is configured for; providing the display data in the form of the live image pixel stream by overlaying the provided live image data with the provided overlay data (Geric ¶0045 “Augmented-reality display 60 may be comprised of a head-worn display, sometimes also referred to as “smart glass” or “smart glasses”, among other names.”, ¶0048 “Augmented-reality display 60 is configured to render glyphs (e.g., text, symbols, colored overlays, etc.) and to render video in the viewfield”, Mixing unit required for superimposing); and not reading the restricted area overlay data (Geric ¶0052 “Restriction of temperature overlay 70 to just the significant locations can be obtained by 1) using pattern recognition…” only relevant surface overlay data is read for mixing /overlaying).
As per claim 6, Geric in view of Guha further teaches wherein the composer device comprises a mixing unit that is configured for providing the display data in the form of the live image pixel stream by overlaying the provided live image data with the provided overlay data, and wherein the composer device is further configured for; discarding the restricted area overlay data (Geric ¶0052 “Preferably, the temperature overlay may be limited to relevant surfaces”, other overlay data is preferably discarded); or determining a transparency value/alpha value of the restricted area overlay data (this is an optional limitation and examiner does not choose this).
As per claim 8, Geric in view of Guha further teaches wherein the composer device is configured for only considering the restricted area overlay data, when an additional security procedure is successfully performed (Geric ¶0052 “Restriction of temperature overlay 70 to just the significant locations can be obtained by 1) using pattern recognition …or 2) by only representing temperatures over a predetermined threshold (e.g., 40° C.)” steps 1 and 2 are security procedures).
As per claim 9, Geric in view of Guha further teaches wherein the additional security procedure comprises a predetermined specific user interaction performed by a user (Geric ¶0063, Fig 19 step 10, manual identification of side branch).
As per claim 10, Geric in view of Guha further teaches wherein the additional security procedure comprises a complex enabling sequence (Geric Fig 19 step 10, ¶0057 “an automated analysis of images is used to identify optimal location … characterized by image analysis (e.g., pattern recognition and depth detection)” Automated pattern recognition is, or contains complex enabling sequences for determining the relevant surfaces for display).
As per claim 13, Geric in view of Guha further teaches wherein the image source comprises a medical imaging device (Geric ¶0003 “devices and methods for endoscopic…”).
As per claim 14, Geric in view of Guha further teaches wherein the overlay data comprises one or more of: sensor data, patient monitoring data, 3D visualisation data, overlay menu data and/or videoconference data (Geric ¶0048, ¶0052).
As per claims 16, 19-20 have limitations similar to claims 1, 5-6 and are rejected for same reasons as above.
Claims 2-4, 17-18 rejected under 35 U.S.C. 103 as being unpatentable over Geric in view of Guha as applied to claims 1, 16 above, and further in view of Raghoebardajal [US 20120262485 A1]
As per claims 2-4, 17-18, Geric in view of Guha does not expressly teach wherein each pixel of the live image pixel stream of the live image data comprises x-y- coordinates, wherein the restricted area of the live image pixel stream of the live image data is determined using predetermined restricted x-y-coordinates, and wherein the restricted area comprises a shape, wherein a dimension and a location of the shape in the region of interest is at least partly defined by the x-y-coordinates, or wherein the restricted area
Raghoebardajal, in the field of processing for augmented reality teaches wherein each pixel of the live image pixel stream of the live image data comprises x-y- coordinates (Raghoebardajal ¶0151 “forward mapping of [W] and [P] applied to the local coordinates x.sub.L, y.sub.L of pixels”),
wherein the restricted area of the live image pixel stream of the live image data is determined using predetermined restricted x-y-coordinates (¶0157 “3020A'', 3020B'' and 3020C'' are restricted by a circular mask. This mask corresponds to the circular buttons 3020A-C seen in the augmented image layer. The masks may be used in one or both of two ways. Firstly, the mask may be used to restrict the mapping of pixels from the control region to the augmentation image layer by use of [W][P],”), and
wherein the restricted area comprises a shape (Raghoebardajal ¶0157 circular shaped masks),
wherein a dimension and a location of the shape in the region of interest is at least partly defined by the x-y-coordinates (Raghoebardajal ¶0129 “The control image has a predetermined size and each of a plurality of control interfaces has a predetermined region within the control image”), or
wherein the restricted area of the live image pixel stream of the live image data is determined using a mask layer (Raghoebardajal ¶0157 “Control regions 3020A'', 3020B'' and 3020C'' are restricted by a circular mask”).
Raghoebardajal provides schemes for interreacting seamlessly with virtual entities for on-screen interplay of live video and augmentation (Raghoebardajal ¶0008). Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the apparatus / method in Geric in view of Guha, by integrating graphical augmentation image layer and plurality of control interfaces and processing as in Raghoebardajal, so as to allow a user of thermal management using AR system, to interact seamlessly with virtual entities.
Claim 11, 21 rejected under 35 U.S.C. 103 as being unpatentable over Geric in view of Guha as applied to claim 10, 16 above, and further in view of Watts [US 9802317 B1].
As per claims 11, 21, Geric in view of Guha does not expressly teach wherein the complex enabling sequence comprises a challenge response from the data server
Watts, in a field of collecting and providing service provider records, teaches wherein the complex enabling sequence comprises a challenge response from the data server
As per MPEP 2143.I.F “Known work in one field of endeavor may prompt variations of it for use in either the same field or a different one based on design incentives or other market forces if the variations are predictable to one of ordinary skill in the art” is a rationale to support obviousness. In the instant case the claim is directed to providing challenge response to a user, which is known to already exist in the field of collection and providing records as in Watts. Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the apparatus in Geric in view of Guha, so as to achieve predictable results of preventing a user from interacting with and/or viewing elements with content that they do not have rights to view.
Claim 15 rejected under 35 U.S.C. 103 as being unpatentable over Geric in view of Guha as applied to claim 1 above, and further in view of Mattila [US 20100328344 A1].
As per claim 15, Geric in view of Guha does not expressly teach wherein the composer device is implemented at least partially in an ASIC or FPGA.
Mattila, in a related field of augmented reality user interface teaches wherein the composer device is implemented at least partially in an ASIC or FPGA (Mattis ¶0047).
As per MPEP 2143.I.A “Combining prior art elements according to known methods to yield predictable results” is a rationale to support obviousness. Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to modify the apparatus in Geric in view of Guha so as to achieve predictable result of implementing via software, hardware using prior art components.
Allowable Subject Matter
Claim 12 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. In combination with parent claims, none of the references of record applied individually or in combination teaches wherein the additional security procedure comprises a request for overlay that enables the composer device to consider the restricted area overlay data for a predetermined limited time and to disable overlay automatically if no additional security procedure is passed before the limited time times out.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OOMMEN JACOB whose telephone number is (571)270-5166. The examiner can normally be reached 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ANNE M KOZAK can be reached at 571-270-0552. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Oommen Jacob/Primary Examiner, Art Unit 3797
Prosecution Insights