Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities:
At [0016], ‘automated recognition objects is used’ should read ‘automated recognition of objects is used’
At [0017], ‘multiple MLMs having different inputs sizes’ should read ‘multiple MLMs having different input sizes’
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: ‘processing devices’ in claim 11.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, in particular an Abstract Idea falling under at least the (c) mental processes grouping (concepts performed in the human mind including an observation, evaluation, judgement, opinion), not ‘integrated into a practical application’ at Prong Two of Step 2A and without ‘significantly more’ at Step 2B.
Regarding claims 1-10:
Step 1: The claims in question are directed to a method comprising “identifying” a portion of an image, “selecting” an MLM based on image portion size, and “processing” the recited image portion using the selected MLM (Step 1: Yes).
Step 2A Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim. Representative claims 1 and 11 recite at a high level of generality – “identifying…a first portion of the first image…selecting, based on a size of the first portion, a first machine learning model (MLM)…processing… the first portion of the first image to detect the presence of one or more moving objects” which is at least a mental process under the Abstract Ideas grouping. as the recited ‘identifying, selecting, and processing’ are not precluded from being performed mentally/visually.
Even if the recited ‘processing using the first MLM’ step requires the use of a computer (e.g. running the model to perform the recited object detection), limitations involving intermediate computerized processing steps do not preclude the ‘processing’ itself from being performed mentally, and the recited ‘detection’, is not of a level of complexity (as recited) that would render the process impossible/impractical for mental evaluation. See MPEP 2106.04(a)(2) subsection C. A Claim that Requires a Computer May Still Recite a Mental Process. Dependent claims are similarly analyzed at Prong One (e.g. claims 2-10) as they further comprise one or more limitations that may similarly be drawn under the mental processes Abstract Idea grouping. For the case of e.g. claim 10, note that although a ‘first camera’ and a ‘second camera’ are recited, limitations involving image acquisition do not preclude the following steps of the method recited in claim 1 from being done mentally (Step 2A, Prong One: Yes).
Step 2A Prong Two: This part of the eligibility analysis evaluates whether the claim integrates the recited judicial exception into a practical application (derived from Alice/Mayo step two and not to be conflated with an assessment of utility– MPEP 2103) of the exception. This evaluation is performed by (1) identifying whether there are any ‘additional elements’ recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim integrates the exception into a practical application. See MPEP 2106.04(d). Examiner notes for consideration at Prong Two of 2A that MPEP 2106.05(a), (b), (c), and (e) generally concern limitations that are indicative of integration, whereas 2106.05(f), (g), and (h) generally concern limitations that are not indicative of integration. As an additional note, ‘additional elements’ are generally limitations excluded from interpretation under the Abstract Idea groupings and may comprise portions of limitations otherwise identified as falling under those Abstract Idea groupings of the 2019 PEG (e.g. any ‘determination’ that may be made mentally accompanied by the use of generic computer hardware considered under the ‘apply it’ considerations of 2106.05(f)). Any ‘providing’/outputting broadly, and ‘collection’ of data (i.e. image acquisition(s)), be they images visually observable/ evaluated by a user/operator, also fail(s) to integrate at least in view of MPEP 2106.05(g) (extra-solution data gathering/output). The same determination holds for dependent claims that serve to limit the collection of data/images (by means of what is collected based on recited conditions) and/or introduce limitations generally linking to a field of use. None of the instant claims appear to explicitly/clearly capture/recite any disclosed improvement in technology (see MPEP 2106.05(a), with note that ‘functioning of a computer’ concerns functions integral to the way a computer operates and not ‘functions’ that a generic computer can be programmed/adapted to perform (see also 2106.05(f))) and any ‘additional elements’, even when considered in combination, fail to integrate at Prong Two of Step 2A accordingly. Integration in view of subsection (a) of 2106.05 requires an identification of the manner in which the improvement is achieved, to be explicitly and specifically (not at a high level of generality i.e. the ‘use of’ complementary modalities/data broadly for any conceivable satellite image analysis not inconsistent with diverse example embodiments) recited in the claims, as ‘additional elements’ precluded from interpretation under any of the Abstract Idea groupings (since the improvement cannot be to the exception itself).
Regarding claims 1-3, 5-7, 9, 11-18, and 20, these claims fail to integrate the use of the recited ‘machine learning model (MLM)’ into a practical application. Although the use of multiple MLMs are recited throughout the incident disclosure, the claims as recited fail to provide information about how the MLM operates to perform the recited method and/or system steps. As such, interpretation using the plain meaning of ‘selecting’ and ‘processing’ does not preclude a human user from conducting the same steps as the MLM, and therefore, does not sufficiently integrate the MLM into a practical application. See below for an excerpt from Example 47 of the 2024 PEG, as the provided interpretation of ‘ANN’ is similar to the incident use of ‘MLM’. Note that “selecting a particular data source or type of data to be manipulated” is an explicit example of an ‘insignificant extra-solution activity’ per MPEP 2106.05(g). Additionally, see below for MPEP 2106.05(a) regarding improvements to computer functioning, noting that the claims as recited can all be performed mentally without unreasonable time or effort.
Regarding claim 3, the recited ‘rescaling’ may qualify as an ‘additional element’, however, in view of the claims as recited, ‘amounts to necessary data gathering and outputting’ according to MPEP 2106.05(g). Per Applicant’s specification at [0017], one goal of the method “reduces the amount of processing involved in rescaling and, for many frames, may completely eliminate the need for rescaling”, disclosing how rescaling only happens when absolutely necessary for proper system functioning, and in many cases, doesn’t even happen in the claimed method at all. As such, an element that only occurs out of necessity for the continuation of the disclosed method fails to integrate the recited exception into a practical application.
Regarding claim 10, the recited ‘first camera’ and ‘second camera’, at [0021] of Applicant’s specification, “Computing device 102 may be configured to receive an image (frame) 101, which may be a part of a stream of images, e.g., a video feed generated by one or more cameras connected to the computing device 102 over any suitable wired or wireless connection… devices capable of generating video feeds (including image 101) can be surveillance cameras, video recorders, photographic equipment, scanners, autonomous vehicle sensing devices (e.g., lidars, radars, long- and mid-range cameras), and the like”. Note that the provided description of camera devices requires only that it be capable of taking video with several example embodiments of video-compatible camera devices. As such, it provides insufficient improvement to the incident technology. Note that per MPEP 2106.05(a), a court-indicated example of insufficient technological improvement is “Delivering broadcast content to a portable electronic device such as a cellular telephone, when claimed at a high level of generality, Affinity Labs of Tex. v. Amazon.com, 838 F.3d 1266, 1270, 120 USPQ2d 1210, 1213 (Fed. Cir. 2016); Affinity Labs of Tex. v. DirecTV, LLC, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016)” which is similarly applicable to the use of video feed capture devices in this case, simply for capturing data to be used in the method/system of the incident independent claims.
Regarding claim 11 and 13-16, the recited ‘one or more processing devices’ add only generic computer components to the proposed method and thus, do not integrate the recited exception into a practical application. Per Applicant’s specification at [0024], “Computing device 102 may include a memory 104 communicatively coupled with one or more processing devices, such as one or more graphics processing units (GPU) 110 and one or more central processing units (CPU) 130”. As this is the only mention of ‘processing device(s)’ in Applicant’s specification and provides only examples of possible processing devices without mention of how they specifically improve the present technology, the ‘processing devices’ in the claims as recited fail to integrate the exception into a practical application. See MPEP 2106.05(a) below for further clarification.
Regarding claim 19, the ‘one or more graphics processing units’ are recited in Applicant’s specification as a potential embodiment where “a GPU 110 includes multiple cores 111, each core being capable of executing multiple threads 112” at [0029]. Note that this is a description of capabilities present in most GPUs, and as such, is considered a ‘generic computer component’ that fails to integrate the exception into a practical application. See MPEP 2106.05(a) below for further clarification.
Regarding claim 20, the recited ‘one or more processors’ add only generic computer components to the proposed method and thus, do not integrate the recited exception into a practical application. Per Applicant’s specification at [0053], “In at least one embodiment, code, such as graph code, loads weight or other parameter information into processor ALUs based on an architecture of a neural network to which such code corresponds”. Note that the recited processor function is one common of processors, and as such is considered a ‘generic computer component’. See MPEP 2106.05(a) below for further clarification.
2024 PEG, Example 47:
The limitations “(a) receiving, at a computer, continuous training data” and “(f) outputting the anomaly data from the trained ANN” are mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity
MPEP 2106.05(a):
It is important to note that in order for a method claim to improve computer functionality, the broadest reasonable interpretation of the claim must be limited to computer implementation. That is, a claim whose entire scope can be performed mentally, cannot be said to improve computer technology.
Examples that the courts have indicated may not be sufficient to show an improvement in computer-functionality:
iii. Mere automation of manual processes, such as using a generic computer to process an application for financing a purchase, Credit Acceptance Corp. v. Westlake Services, 859 F.3d 1044, 1055, 123 USPQ2d 1100, 1108-09 (Fed. Cir. 2017) or speeding up a loan-application process by enabling borrowers to avoid physically going to or calling each lender and filling out a loan application, LendingTree, LLC v. Zillow, Inc., 656 Fed. App'x 991, 996-97 (Fed. Cir. 2016) (non-precedential)
It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981))
To show that the involvement of a computer assists in improving the technology, the claims must recite the details regarding how a computer aids the method, the extent to which the computer aids the method, or the significance of a computer to the performance of the method. Merely adding generic computer components to perform the method is not sufficient. Thus, the claim must include more than mere instructions to perform the method on a generic component or machinery to qualify as an improvement to an existing technology
MPEP 2106.05(g):
This consideration is similar to factors used in past Office guidance (for example, the now superseded Bilski and Mayo analyses) that were described as mere data gathering in conjunction with a law of nature or abstract idea. When determining whether an additional element is insignificant extra-solution activity, examiners may consider the following:
(3) Whether the limitation amounts to necessary data gathering and outputting, (i.e., all uses of the recited judicial exception require such data gathering or data output). See Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015) (presenting offers and gathering statistics amounted to mere data gathering). This is considered in Step 2A Prong Two and Step 2B
Even when viewed in combination, the ‘additional elements’ present (namely the use of generic computer/technology components, insignificant extra-solution activity, and necessary data gathering and outputting) do not integrate the recited judicial exception into a practical application, the claims are directed to the judicial exception (Step 2A Prong Two: No).
Step 2B: This part of the eligibility analysis evaluates whether the claim amounts to ‘significantly more’ than the recited exception, i.e., whether any ‘additional element’, or combination of additional elements, adds an inventive concept to the claim. The considerations of Step 2A Prong 2 and Step 2B overlap but differ in that 2B also requires considering whether the claims/limitations feature any “specific limitation(s) other than what is well-understood, routine, conventional activity in the field” (WURC) (MPEP 2106.05(d)). Such a limitation if specifically recited however, must still be excluded from interpretation under any of the Abstract Idea groupings (else it is not an ‘additional element’ and extra-solution activity need not then be implicated). Limitations precluded from serving as indications of an inventive concept/ ‘significantly more’ include those that are not specifically recited (instead recited at a high level of generality), those that are established as WURC, those that are not ‘additional elements’ by nature of their analysis at Prong One (i.e. directed to the exception), and/or those that otherwise fail to serve for integration at Prong Two of 2A in view of 2106.05(f), (g) and/or (h). The claim(s) in question recite little beyond those limitations recited at a high level of generality and falling under the exception. (Step 2B: No).
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over one or more claims (see correspondence tables below) of U.S. Patent No. 12,211,216 B2 to parent application 17/584,151 in view of references of record to include Park (Multiresolution Models for Object Detection) and Bhat (US20210110552A1). Although the claims at issue are not identical, they are not patentably distinct from each other because claims of reference anticipate and/or render obvious independent claims of the instant application, in further view of the following reasons/considerations:
• Instant claims and claims of reference recite common subject matter, and recite the open ended transitional phrase “comprising” which does not preclude any additional elements recited by claims of reference – see the limitation mappings/table presented below;
• Language/terminology of instant claim(s) constituting minor/slight variations from the claims of reference, if/where present (e.g. ‘portion vs. ‘region’ of an image, ‘bounding box’ vs. ‘bounding shape’, etc.), require interpretations under Broadest Reasonable Interpretation and/or plain meaning definitions (MPEP 2173 and 2111) equivalent to/met by language of the reference claims in view of that corresponding/shared Specification. While the disclosure of reference may not be used as prior art (Double Patenting concerns the claims of reference), portions of the specification which provide support for reference claims may also be examined and considered when addressing the scope of claim(s) of reference and the issue of whether an instant claim defines an obvious variation or falls within the scope of an invention claimed in the claim(s) of reference. See MPEP 804 with reference to In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970).
• Instant claims that differ merely in statutory category (computer implemented method vs. a system comprising processing device(s) vs. processor(s) to carry out the method), if otherwise congruent in scope (e.g. instant claims 1/11/20), may be rejected in view of Obviousness type Double Patenting procedures as they relate to system/processor/method claims of reference with congruent scopes.
• Whereby elements of instant claims otherwise not present explicitly in corresponding reference claims (see * in table below – namely that a ‘second size of a second portion’ corresponds to limitations evidenced by literature of record (see e.g. the corresponding prior art-based rejections below) and requiring no more than obvious modification to the claims of reference. Examiner notes that although not explicitly stated in recited claim language of reference, the proposed method is capable of detecting multiple objects in the same image, as in the reference specification of US 12,211,216, it is disclosed that “the trained OCM(s) 126 may be deployed to classify various objects within image 101”. This disclosure, in addition to the prior art as outlined below under 35 U.S.C 103 rejections, evidences the obvious nature of the limitations recited in the incident claims. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date to modify claims of reference in a manner consistent with those modifications and supporting rationale provided in the prior-art based rejections below in addition to the evidence provided herein.
Instant Application
Claims of Reference
(US 12,211,216 B2)
Claim 1/11: A method comprising:/ A system comprising:
identifying, using a plurality of motion vectors characterizing displacement of a content of a first image relative to a reference image, a first portion of the first image depicting one or more moving objects
Claim 1/11: A method comprising: / A system comprising:
identifying a first image and a first reference image; determining a first plurality of motion vectors (Vs), each of the first plurality of MVs characterizing displacement of one or more pixels of the first image relative to the first reference image; identifying, using the first plurality of MVs, a first region of the first image, the first region comprising a depiction of one or more objects in the first image
Claim 1/11/20: selecting, based on a size of the first portion, a first machine learning model (MLM) of a plurality of MLMs, each of the plurality of MLMs associated with a respective one of a plurality of input sizes
Claim 1/11: selecting, based on a size of the first region, a first machine learning model (MLM) of a plurality of MLMs, each of the plurality of MLMs corresponding to a respective one of a plurality of input sizes
Claim 1/11/20: and processing, using the first MLM, the first portion of the first image to detect presence of the one or more moving objects in the first portion of the first image
Claim 1/11: selecting, based on the size of the first region and an input size corresponding to the first MLM, a first portion of the first image, wherein the first portion comprises the first region; and detecting a presence of the one or more objects in the first image based on an output of processing of the first portion using the first MLM
Claims 2/3/4
Claims 2/3/4
Claims 5/14
*see bullet 4 under Obviousness type Double Patenting above in view of references Park and Bhat
Claims 6/8/9/10/19
Claim 6/7/8/9/10
Claim 7
Claims 14/19
Claim 12
Claim 6
Claims 13/15/16
Claims 13/15/14
Claims 17/18
Claims 7/8
Claim 20 (i.e. the ‘one or more processors’ claim of the method and system disclosed in claims 1/11 above)
See above (bullet 3 under Double Patenting), where claims of different statutory categories with otherwise congruent scope are rejected due to Obviousness under Double Patenting
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Park (Multiresolution Models for Object Detection) in view of Bhat (US20210110552A1).
Regarding claim 1, Park teaches a method comprising: selecting, based on a size of the first portion, a first machine learning model (MLM) of a plurality of MLMs (“In this paper, we define a feature representation Φ(x) that directly processes windows of varying size” [page 243], noting that a “method of dealing with windows of varying sizes is to build a separate model for each size. Assume that every window x arrives with a bit s that specifies whether it is “small” or “large”” [page 244]) each of the plurality of MLMs associated with a respective one of a plurality of input sizes ("Assume that every window x arrives with a bit s that specifies whether it is "small" or "large”…it is equivalent to partitioning the dataset into small and large instances and training on each independently" [page 244], noting that 'window' is a portion of an image, see 243) and processing, using the first MLM, the first portion of the first image to detect presence of the one or more moving objects in the first portion of the first image. (see Fig. 3 below, noting that “On the left, we show the result of our low-resolution rigid-template baseline. One can see it fails to detect large instances. On the right, we show detections of our high-resolution, part-based baseline, which fails to find small instances” [page 248]). Park falls silent regarding a method comprising: identifying, using a plurality of motion vectors characterizing displacement of a content of a first image relative to a reference image, a first portion of the first image depicting one or more moving objects.
PNG
media_image1.png
360
884
media_image1.png
Greyscale
However, Bhat teaches a method comprising: identifying, using a plurality of motion vectors characterizing displacement of a content of a first image relative to a reference image, a first portion of the first image depicting one or more moving objects (“The example motion vector object detection analyzer 210 compares the position of the matching pixels blocks in each of the two image frames to determine the displacement of the matching pixels between two image frames (e.g., number of pixels shifted in either the X or Y direction in the image frames). The motion vector object detection analyzer 210 generates a motion vector based on the X and Y displacement of the matching pixels between the two image frames within" (column 4, [0037]), note that per Fig. 8, the 'plurality' of motion vectors can be seen when the above process is performed on several pixels and the results are aggregated).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the object detection multiresolution models as disclosed by Park to include the moving object detection capabilities as disclosed by Bhat, the suggested motivation being to allow for differentiation between moving foreground objects and still background portions between image frames to eliminate the need for object detection processing steps on the entire image in applications where the user is interested in foreground objects, in, for example roadside surveillance situations.
Regarding claim 2, Park in view of Bhat teaches the method of claim 1. Park further teaches wherein an input size associated with the first MLM matches a size of the first portion of the first image ("the simplest method of dealing with windows of varying sizes is to build a separate model for each size" [page 244])
Regarding claim 3, Park in view of Bhat teaches the method of claim 1. Park further teaches wherein an input size associated with the first MLM is different from a size of the first portion of the first image ("there will be a collection of training instances of "intermediate" size that could be processed as low or high-resolution instances" [page 246]) further comprising: rescaling, prior to processing the first portion of the first image using the first MLM, the first portion of the first image to match the input size associated with the first MLM ("all instances are either 50 or 100 pixels tall. Given a training window of arbitrary height xi, one might resize it to 50 or 100 pixels" [page 246]).
Regarding claim 4, Park in view of Bhat teaches the method of claim 1. Park further teaches wherein detecting presence of the one or more objects in the first image comprises determining a bounding shape for at least one object of the one or more objects in the first image (see one instance of Fig. 3 below, noting that the ‘multiresolution model’ is the proposed model by Park).
PNG
media_image2.png
366
449
media_image2.png
Greyscale
Regarding claim 5, Park in view of Bhat teaches the method of claim 1. The proposed combination also teaches selecting, based on a second size of the second portion, a second MLM of the plurality of MLMs and processing, using the second MLM, the second portion of the first image to detect presence of the one or more additional (Note that the above disclosure was included in the rejection for claim 1, and that the ‘plurality of MLMs’ referenced in Park can be directed to the same aspects of the disclosure as already included herein). As previously indicated above in the rejection of claim 1, Park fails to explicitly disclose identifying, using a plurality of motion vectors, image portions depicting one or more moving objects.
Bhat, however, discloses further comprising: identifying, using the plurality of motion vectors, a second portion of the first image depicting one or more additional moving objects (see Figs. 7 and 8 below, where multiple moving objects are being detected from the same image). Note also that Bhat discloses specifically ‘moving’ objects and the multiple portions of a captured image, as visible in Figs. 9 and 10 below, note the bounding boxes representing distinct image portions in Fig. 10.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the proposed combination in view of Bhat regarding using motion vectors to identify a second portion of a first image, the proposed motivation being that an object detection model that could process multiple moving objects of different sizes from the same image frame would have increased functionality compared to the previously proposed combination.
PNG
media_image3.png
356
544
media_image3.png
Greyscale
PNG
media_image4.png
355
524
media_image4.png
Greyscale
Regarding claim 6, Park in view of Bhat teaches the method of claim 1. Bhat further teaches determining, using one or more classifier MLMs, a type of at least one object of the one or more moving objects (“Outputs of the camera(s) 104 are provided to an example vision data analysis system 106 that implements the data processing functionality of a vision-based driver-assistance system to detect and annotate recognized objects in the surrounding environment based on an AI vision- based analysis of images captured by the camera(s)” [0028], see Figs. 12 and 13 for visual of object moving between frames, and note that “the Al-based boundary boxes in the first, fourth, and fifth image frame in FIG. 13 are also annotated with a label (e.g., "BIKE”) to classify the object) based at least on processing a combined input (“Once a motion vector boundary box that matches a corresponding Al - based boundary box in a particular image frame has been annotated with a corresponding label, the image frame and all meta information associated with the image frame (e.g. a timestamp, boundary box coordinates, associated labels, etc.) are stored in the image data database 204. Thereafter, the example boundary box analyzer 212 may continue to analyze the image frame based on additional motion vector boundary boxes to be compared with other Al-based boundary boxes in the image frame for classification” [0052]) that includes: a first output of processing the first portion of the first image using the first MLM (“a boundary box analyzer to determine whether the motion vector boundary box corresponds to any artificial intelligence (AI)-based boundary box generated based on an analysis of the first image6 using an object detection machine learning model” [0083]), and a second output of processing a second portion of a second image using at least one of the first MLM (“the boundary box analyzer to determine that the second motion vector boundary box corresponds to an Al-boundary box generated based on an analysis of the second image” [0092]) or a second MLM of the plurality of MLMs (Examiner notes that Bhat satisfies the first MLM of the ‘at least one’ MLMS, but additionally that Park discloses a ‘second MLM of the plurality of MLMs’ should the necessity for such clarification arise).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the proposed combination in view of Bhat regarding determining the proposed input based on a combined input, the suggested motivation being that using an aggregation of object detection determinations rather than just one increases the likelihood of a correct classification, especially in moving object applications.
Regarding claim 7, Park in view of Bhat teaches the method of claim 6. Beyond this, claim 7 recites identical limitations to claim 1, substituting the ‘second portion of the second image’ as disclosed in claim 6 for the ‘first portion of the first image’ disclosed in claim 1. As the method is identical as described regardless of which image is being processed, claim 7 is rejected according to claim 1, modified to include the ‘second portion of the second image’ as disclosed above. See claims 1 and 6 for limitation rejections.
Regarding claim 8, Park in view of Bhat teaches the method of claim 6. Bhat teaches wherein the one or more moving objects comprise a vehicle, and wherein processing the combined input is further to determine one or more of: a type of the vehicle (see Fig. 14 below, where the moving object is a bike and is detected and labelled accordingly), a make of the vehicle, or a model of the vehicle (Examiner notes that ‘one or more’ is satisfied by this interpretation, and that a bicycle falls under the broadest reasonable interpretation of ‘type of vehicle’). Note also that per [0028] of Bhat, the model can differentiate between e.g. ‘car’, ‘bus’, and truck’, and per [0052], between e.g. ‘car’ and ‘motorcycle’.
PNG
media_image5.png
289
813
media_image5.png
Greyscale
Regarding claim 9, Park in view of Bhat teaches the method of claim 6. Bhat further teaches wherein the combined input is one of a plurality of combined inputs (“If multiple Al-based boundary boxes were generated at block 1506, the example boundary box analyzer 212 may compare the motion vector boundary box generated at block 1508 to each of the different Al-based boundary boxes (at least until a match is detected). The example boundary box analyzer 212 performs analysis on one pair of boundary boxes (motion vector boundary box and an Al-based boundary box) of a selected image frame at a time” [0066]. Note that the example input combination is one of many options, and several can be combined during the object detection process), and wherein the one or more classifier MLMs perform pipelined processing of the plurality of combined inputs (“the index may be a running number index that increases with each subsequent image frame in which a motion vector boundary box is generated that does not match with a corresponding Al based boundary box” [0057], noting that “The example boundary box analyzer 212 performs analysis on one pair of boundary boxes (motion vector boundary box and an Al-based boundary box) of a selected image frame at a time” [0066]. Note additionally that “one or more of the elements, processes and/or devices illustrated in FIG. 2 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way” [0060]) in further view of that modification as proposed above for the case of claim 6.
Regarding claim 10, Park in view of Bhat teaches the method of claim 6. Bhat further teaches wherein the first image is obtained using a first camera and the second image is obtained by a second camera (“the vehicle 102 includes one or more camera(s) 104 to capture image data indicative of objects and/or conditions in a surrounding environment” [0028], and “the first and second image frames correspond to adjacent or successive image frames in a video stream or series of images captured by the camera(s) 104 with the second image frame being captured at a later point in time than the first image frame [0037]. Examiner notes that while it is not a requirement in the disclosure by Bhat to use multiple cameras capturing multiple fields of view, the disclosure as recited includes that instance as a possible implementation), and wherein a field of view of the first camera is different from a field of view of the second camera (“The image data is representative of a view of the environment that is in front of the example vehicle 102, to the side of the vehicle 102, to the rear of the vehicle 102, and/or in any other direction” [0031]. Examiner notes that while it is not a requirement in the disclosure by Bhat to use multiple cameras capturing multiple fields of view, the disclosure as recited includes that instance as a possible implementation).
Claim 11 is the system claim corresponding to the method of claim 1 and is rejected accordingly. See below (re claim 19 rejection) for rationale regarding the processors that are necessary for the system to carry out the method as disclosed in claim 1. Also note that Bhat further discloses a system in addition to the disclosed method (see Fig. 1, e.g.).
Claims 12 and 13, dependent on claim 11, are the system claim counterparts to the method claims 2 and 3 (respectively), dependent on claim 1, and are rejected accordingly. See claims 2, 3, and 11 for limitation rejections.
Claims 14 and 15, dependent on claim 11, discloses the same embodiments as claims 5 and 6 (respectively), dependent on claim 1, and is rejected accordingly. Examiner notes that although claims 15 and 16 disclose processing devices absent from claims 5 and 6, claims disclosing the same inventive concept of/with different statutory categories are rejected accordingly. See claims 5 and 6 for limitation rejections and see claim 20 regarding the presence of processing devices present in the prior art capable of performing the system as disclosed.
Claim 16, dependent on claim 15, discloses the same embodiments as claim 7, dependent on claim 6, and is rejected accordingly. Examiner notes that although claim 16 discloses processing devices/processing absent from claim 6, claims disclosing the same inventive concept of/with different statutory categories are rejected accordingly. See claim 7 for limitation rejections and see claim 20 regarding the presence of processing devices present in the prior art capable of performing the system as disclosed.
Claim 17, dependent on 15, discloses the same embodiments as claim 8, dependent on claim 6, and is rejected accordingly. Examiner notes that although claim 17 discloses processing absent from claim 9, claims disclosing the same inventive concept of/with different statutory categories are rejected accordingly. See claim 8 for limitation rejections and see claim 20 regarding the presence of processing devices present in the prior art capable of performing the system as disclosed.
Claim 18, dependent on claim 15, discloses the same embodiments as claim 9, dependent on claim 6, and is rejected accordingly. Examiner notes that although claim 18 discloses processing devices/processing absent from claim 9, claims disclosing the same inventive concept of/with different statutory categories are rejected accordingly. See claim 9 for limitation rejections and see claim 20 regarding the presence of processing devices present in the prior art capable of performing the system as disclosed.
Regarding claim 19, Park in view of Bhat teaches the system of claim 11. Bhat further teaches wherein the one or more processing devices comprise one or more graphics processing units (“Thus , for example , any of the example camera interface 202, the example image data database 204, the example video decoder 206, the example AI vision-based driver-assistance system analyzer 208, the example motion vector object detection analyzer 210, the example boundary box analyzer 212, the example server interface 214 and/or, more generally, the example vision data analysis system 106 could be implemented by one or more…graphics processing unit(s) (GPU(s))” [0060], noting that any of the above listed system components are ‘processing devices’ when taking the broadest reasonable interpretation of the term).
Claim 20 is the processor claim corresponding to the method of claim 1 and is rejected accordingly. See column 9, section [0061] of Bhat for disclosure that necessitates a processor to carry out method steps as disclosed in claim 1.
Additional References Cited
Prior art made of record and not relied upon that is considered pertinent to applicant’s disclosure:
Additionally cited references (see attached PTO-892) otherwise not relied upon
above have been made of record in view of the manner in which they evidence the general state of the art. Examiner notes that Dwivedi (US 11,568,625), supplied by Applicant on IDS, discloses similar content with different named inventors, sharing common ownership under NVIDIA Corporation. While it was not applied as prior art for this reason, Examiner notes that it demonstrates limitations characteristic of current state of the art as well as limitations that may have been applied as prior art regarding content and excluding exceptions under 35 U.S.C. 102(b)(2). Additionally, examiner notes that Uzkent (Efficient Object Detection in Large Images using Deep Reinforcement Learning) teaches content characteristic of the current state of the art and may alternatively be applied as prior art regarding at least some limitations of Applicant’s disclosure. All other cited references may similarly/alternatively serve to anticipate at least the independent claims as recited, and/or provide examples of current art to Applicant as discovered and noted by Examiner during search.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA T RAMETTA whose telephone number is (571)272-0451. The examiner can normally be reached Monday- Friday, 8 a.m. 5 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JULIA T RAMETTA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights