DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim(s) 1, 8, and 15 recite “to an unknown object in one of the classes of objects.” It is unclear how an object can be unknown and be in a known class. For the purpose of examination, this phrasing is interpreted to mean that an object falls within the semantic region or category of a known class but does not match a specific class definition.
Claim(s) 2-7, 9-14, and 16-20 depend either directly or indirectly from the rejection of Claim(s) 1, 8, and 15, therefore they are also rejected.
Appropriate correction is required.
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “device configured to” in claim(s) 8-14.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to abstract ideas without significantly more. The claim(s) recite(s) mathematical concepts, mathematical calculations, and mental process/judgment. This judicial exception is not integrated into a practical application because the claims merely output the result of mathematical calculations without any practical application of that output. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because data gathering (receiving embeddings, receiving query) and outputting (generating output signal) are well-understood, routine, conventional activities.
Step 1: YES - Process (method)
Step 2A Prong 1: Does the claim recite a judicial exception?
The claim recites:
"projecting the embeddings into a hyperbolic embedding space" – Mathematical concept (exponential mapping formula [0067])
"regularizing the projected embeddings ... by moving each ... closer to ... same category and further away from different categories" - Mathematical concept (hyperbolic contrastive loss calculations [0069]-[0072])
"generating ... an output signal that indicates whether the object ... corresponds to an unknown object" - Mental process/judgment + mathematical calculation (distance comparison)
Conclusion: YES - Recites mathematical concepts and mental processes
Step 2A Prong 2: Does the claim integrate into a practical application?
Additional Elements:
"receiving object data" - data gathering (insignificant extra-solution)
"receiving an unmatched query corresponding to an object in a second input image" - data gathering
"generating ... an output signal" - outputting result
Improvement Analysis: The specification describes an improvement to OWOD technology
by modeling hierarchical relationships via hyperbolic geometry, enabling better unknown
detection. However, claim 1 does NOT require:
Actually controlling any device based on the output
Any specific DNN architecture
The full pipeline (synthesizing/acting on results)
The claim ends with "generating ... an output signal." This is merely outputting the result of
mathematical calculations without any practical application of that output.
Compare to: USPTO Example 47 (Anomaly Detection):
Claim 2 (ineligible): Ends with "outputting the anomaly data"
Claim 3 (eligible): Includes remediation steps (dropping packets, blocking traffic)
Claim 1 here is analogous to Example 47 Claim 2 - it recites mathematical calculations and outputs the result without integrating into a practical application.
Conclusion: NO - Does not integrate into practical application
Step 2B: Do additional elements provide significantly more?
The data gathering (receiving embeddings, receiving query) and outputting (generating output signal) are WURC activities. The use of a "hyperbolic embedding space" is merely a field of use limitation.
Conclusion: NO - Does not provide inventive concept
Claims 8-14 Analysis:
Same abstract idea implemented on generic "processing device configured to execute
instructions" - mere instructions to apply exception on generic computer. Ineligible.
Claims 15-20 Analysis:
These include "an actuator configured to control an operation of the computer-controlled
machine in response to the output signal."
This is analogous to Example 47 Claim 3 - the output is used to control actual machine
operation. The specification at [0088]-[0117] describes control of vehicles, robots,
manufacturing machines.
Step 2A Prong 2 for Claim 15: The claim integrates the mathematical analysis into a practical application by:
Using sensor input to generate images
Performing the hyperbolic embedding analysis
Controlling an actuator based on the output
This reflects an improvement to machine control by enabling detection of unknown objects
for operational decisions.
Conclusion: Claims 15-20 are ELIGIBLE - integrate into practical application of machine
control.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Lang et al. (On Hyperbolic Embeddings in Object Detection – hereinafter “Lang”) in view of Ge et al. (Hyperbolic Contrastive Learning for Visual Representations beyond Objects – hereinafter “Ge”).
Claims 1, 8, and 15.
Lang teaches a computer-controlled machine (Lang: "system with an Intel Xenon ... and NVIDIA TITAN RTX GPUs" (p. 8, "Training Protocol").), comprising:
at least one sensor configured to generate a first input image and a second input image (Lang discloses object detection on "complex traffic scenes" (p. 7, "Datasets") and searching for "image regions that potentially contain objects" (p. 1, § 1 ), implying imaging sensors. Lang further discloses processing "an input image" (p. 1, § 1) to "search for image regions that potentially contain objects" (p. 1, § 1 ).);
a control system configured to receive object data, wherein the object data includes embeddings data corresponding to a plurality of embeddings for known objects in a first input image (Lang discloses extracting "encoded visual features of an input image" (p. 1, § 1) and "visual feature vectors vi “ ... from latent image features (p. 4, § 3). Lang further discloses using "ResNet-encoded image features" (p. 7, § 3.3) and "key elements are of the output feature maps" (p. 7, § 3.3) for known objects in the training set (p. 8, "Training Protocol").);
project the embeddings into a hyperbolic embedding space, wherein the hyperbolic embedding space includes embeddings in a plurality of categories of objects, and wherein each of the plurality of categories of objects includes one or more classes of objects (Lang discloses a "classification head ... to learn hierarchical representations of visual features ... in hyperbolic space" (p. 2, § 1 ). Lang teaches "transform visual features v ∈ R n+1 ... into points on the hyperboloid ... apply the exponential map" (p. 5, § 3.1 ). Lang observes that in this space, "categorical class hierarchies emerging" (p. 1, Abstract) and qualitative results show a "living thing neighborhood and a neighborhood of frequently co-occurring classes" (p. 12, § 4.3).);),
regularize the projected embeddings within the hyperbolic embedding space, wherein (Lang discloses that "hyperbolic geometry better matches the underlying structure ... resulting in lower classification errors" (p. 1, § 1 ).) ,
receive an unmatched query corresponding to an object in a second input image (Lang uses "validation images" (p. 8, § 4.1) and evaluates "Zero-shot object detection" (p. 7, "Evaluation metrics") to "recognize unseen object types" (p. 10, "Zero-shot evaluation"), which are unmatched queries.), and generate, based on the hyperbolic embedding space including the regularized embeddings, an output signal that indicates whether the object in the second input image corresponds to an unknown object in one of the classes of objects (Lang’s hyperbolic classifier "outputs the classification logits computed in the learned hyperbolic metric space, i.e. calculates hyperbolic distances" (p. 2, Fig. 1 Caption). Lang generates "precision ... on seen as well as unseen classes" (p. 11, Table 4), which is the claimed output signal.); and
PNG
media_image1.png
795
794
media_image1.png
Greyscale
an actuator configured to control an operation of the computer-controlled machine in response to the output signal (Lang evaluates on "complex traffic scenes" (p. 7, "Datasets"), relevant to "Intelligent Vehicles" (p. 14, Ref. 1) and “robotics” (p. 16, Ref. 36). It would be obvious to use Lang's detection system to control a vehicle's actuator. Where, and “self-driving cars” require sensors (cameras/LiDAR) and actuators (steering/motors) to operate. Lang also disclose generating output signals (classification scores) based on distances in the hyperbolic embedding space (p. 2, Fig. 1; ).).
Lang discloses all of the subject matter as described above except for specifically teaching “regularizing the projected embeddings within the hyperbolic embedding space includes moving each of the projected embeddings (i) closer to embeddings in a same category of the plurality of categories and (ii) further away from embeddings in different categories of the plurality of categories.” However, Ge in the same field of endeavor teaches regularizing the projected embeddings within the hyperbolic embedding space includes moving each of the projected embeddings (i) closer to embeddings in a same category of the plurality of categories and (ii) further away from embeddings in different categories of the plurality of categories (While Lang relies on standard losses, Ge explicitly teaches a "hyperbolic contrastive learning (HCL) framework" (p. 4, Fig. 2) to enforce such structure. Ge discloses using a "hyperbolic loss ... to encourage representations ... to lie close to representations of their constituent objects in a hyperbolic space" (p. 1, Abstract). Ge states the method is trained to "reduce the distance between positive pairs and push away the negative pairs in a hyperbolic space" (p. 2, § 1 ).).
Therefore, it would have been obvious to one of ordinary skill in the art to combine Lang and Ge before the effective filing date of the claimed invention. The motivation for this combination of references would have been to modify the hyperbolic object detector of Lang by incorporating the regularization technique of Ge. While Lang utilizes hyperbolic space to capture latent hierarchies (Lang, p. 1), Ge teaches the active mechanism to enforce this structure: using a hyperbolic contractive loss to regularize “child” embeddings (objects) so that they cluster tightly around their “parent” embeddings (scenes) (Ge, p. 1). A skilled artisan would recognize that Ge’s scene-to-object hierarchy is structurally identical to the claimed category-to-class hierarchy. Thus, applying Ge’s regularization to Lang’s system forces classes to cluster within their parent categories, thereby improving the semantic separation of the embeddings.
Claim 2.
The combination of Lang and Ge method of claim 1, wherein the object data includes a set of bounding boxes and class labels for the known objects in the first input image (Lang: "matching the ground truth bounding boxes" (p. 3, § 2); "labeled instances" (p. 7, "Datasets").).
Claim 3.
The combination of Lang and Ge discloses the method of claim 1, wherein the first input image corresponds to a training image (Lang: "trained on 122,000 training images" (p. 8, "COCO 2017 benchmark").).
Claim 4.
The combination of Lang and Ge discloses the method of claim 1, further comprising determining a hyperbolic contrastive loss corresponding to the hyperbolic embedding space, wherein regularizing the projected embeddings includes regularizing the projected embeddings based on the hyperbolic contrastive loss (Ge: "propose a hyperbolic contrastive objective" (p. 2, § 1 ).).
Claim 5.
The combination of Lang and Ge discloses the method of claim 1, wherein regularizing the projected embeddings includes determining respective hyperbolic averages of embeddings in each of the classes of objects (Lang: "learned class prototypes" (p. 4, § 3) to which distances are calculated.).
Claim 6.
The combination of Lang and Ge discloses the method of claim 5, wherein generating the output signal includes (i) determining a threshold distance based on the hyperbolic averages and (ii) generating the output signal based in part on the threshold distance (Lang: "scaling-hyperparameter dmin" (p. 6, Eqn. 7) defines the decision boundary.).
Claim 7.
The combination of Lang and Ge discloses the method of claim 6, wherein generating the output signal includes determining whether the unmatched query is less than the threshold distance from at least one of the hyperbolic averages (Lang: "computes logits by shifting the distances" (p. 6, Eqn. 7); smaller distance yields higher score/detection.).
Claims 9-14.
The limitations in claims 9-14 correspond to Claims 2-7 and are rejected over Lang and Ge, mutatis mutandis.
Claims 16.
The combination of Lang and Ge discloses the computer-controlled machine of claim 15, further comprising memory that stores data corresponding to the hyperbolic embedding space (Lang learns "class prototypes in hyperbolic space" (p. 4, § 3). Ge uses "a memory bank to
store the negative representations" (p. 4, § 2.2.2).).
Claim 17.
The combination of Lang and Ge discloses the computer-controlled machine of claim 15, wherein the control system is further configured to generate a hyperbolic contrastive loss corresponding to the hyperbolic embedding space, wherein regularizing the projected embeddings includes regularizing the projected embeddings based on the hyperbolic contrastive loss (Ge discloses a "hyperbolic contrastive objective" (p. 2, § 2.1) where " nodes near the root, i.e. objects, close to the center to achieve an overall lower loss" (p. 4, § 2.2.2).).
Claim 18.
The combination of Lang and Ge discloses the computer-controlled machine of claim 15, wherein, to regularize the embeddings, the control system is further configured to determine respective hyperbolic averages of embeddings in each of the classes of objects (Lang uses "learnable class prototypes" (p. 1, § 1) and calculates "distances ... to the learned class prototypes on the hyperboloid. " (p. 2, Fig. 1 ). Prototypes are centroids/averages in metric learning.).
Claim 19.
The combination of Lang and Ge discloses the computer-controlled machine of claim 18, wherein, to generate the output signal, the control system is further configured to (i) determine a threshold distance based on the hyperbolic averages and (ii) determine whether the unmatched query is less than the threshold distance from at least one of the hyperbolic averages (Lang uses a "scaling-hyperparameter dmin" which "defines the distance that
accounts for a classification confidence of p = 0.5" (p. 6, § 3.2). This acts as a threshold.
Lang's "Zero-shot" metrics rely on "thresholds for loU" and scores (p. 7, "Evaluation
metrics").
Claim 20.
The combination of Lang and Ge discloses the computer-controlled machine of claim 15, wherein the computer-controlled machine includes an autonomous robot (Lang evaluates on "complex traffic scenes" (p. 7, "Datasets"), applicable to “robotics” (p. 16, Ref. 36).).
Conclusion
The prior art made of record but not relied, yet considered pertinent to the applicant’s disclosure, is listed on the PTO-892 form.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ross Varndell whose telephone number is (571)270-1922. The examiner can normally be reached M-F, 9-5 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached at (313)446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ross Varndell/Primary Examiner, Art Unit 2674