Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claims 1, 11, and 20, the phrase "e.g. pixels" renders the claim indefinite because it is unclear whether the limitations following the phrase are part of the claimed invention. Examiner notes that under broadest reasonable interpretation, a ‘portion’ of an image does not necessitate the identification of ‘pixels’. Because of this, it is unclear whether Applicant is claiming segmentation by identifying image ‘portions’ or segmentation regarding specifically image ‘pixels’. See MPEP § 2173.05(d) for further explanation.
Regarding claims 3 and 13, the recited “wherein said identifying the one or more objects comprises enabling an open vocabulary event segmentation process” as recited would be unclear to a person of ordinary skill in the art, as it fails to establish under which conditions an open vocabulary event segmentation process would be enabled and when the recited ‘identifying’ of objects does/does not meet the conditions. Even when considering Applicant’s specification, little information is provided regarding how the model is trained and under which circumstances the segmentation on an open vocabulary of objects or events is ‘enabled’.
Regarding claims 7 and 17, the limitation recited is not consistent with the claim on which it depends. Claim 7 (and similarly 17) recites, “The method of claim 1 (11), wherein said identifying the one or more objects and said identifying the event occur in parallel”, but claims 1 and 11 recite “identifying an event based on the living entity and the one or more objects”. Examiner notes that claim 7 as recited seems to require that two processing steps happen at the same time, while claim 1 requires that one of the steps be based on another. Because of this, it would be unclear to a person of ordinary skill in the art how to perform the recited processing. Regarding Applicant’s specification at [0052], “Some implementations, determine both (a) one or more events (e.g., major events) occurring in a video and (b) which objects are involved with each of those events. The events and objects may be determined in parallel within a machine learning model”. Examiner notes that interpreting claim 7 based on this aspect of the disclosure implies that the processing of determining major events and grouping objects based on these events, which would not be inconsistent with the recited limitations at claim 1, since it suggests that initial object detection may have occurred prior for the event to be ‘based on’. For the purposes of compact prosecution, Examiner reads the language “said identifying the one or more objects and said identifying the event” as identifying that occurs after an initial object detection has been conducted, given Applicant’s specification above.
Dependent claims 2, 4-6, 8,10, 12, 14-16, and 18 inherit and fail to cure those deficiencies identified above for independent/intervening claims and are rejected accordingly.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, in particular an Abstract Idea falling under at least the (c) mental processes grouping (concepts performed in the human mind including an observation, evaluation, judgement, opinion), not ‘integrated into a practical application’ at Prong Two of Step 2A and without ‘significantly more’ at Step 2B.
Regarding claims 1-10:
Step 1: The claims in question are directed to a method comprising a device with a processor for “obtaining” video frames, “identifying” objects in the frames, “identifying” an event based on the subjects in the frame, and “segmenting” the frames by identifying portions containing subjects of interest (Step 1: Yes).
Step 2A Prong One: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04, subsection II, a claim “recites” a judicial exception when the judicial exception is “set forth” or “described” in the claim. Representative claims 1, 11, and 20 recite at a high level of generality –
1) “obtaining one or more frames of a video, 2) identifying…objects depicted in the…frames, 3) identifying an event, 4) identifying the subset of the one or more objects, 5) segmenting…comprising identifying portions” which is at least a mental process under the Abstract Ideas grouping. as the recited ‘obtaining, identifying, and segmenting’ are not precluded from being performed mentally/visually.
Even if the recited steps are conducted using ‘an electronic device having a processor’, limitations involving intermediate computerized processing steps do not preclude the ‘processing’ itself from being performed mentally, and the recited ‘segmentation’, is not necessitated to be performed at a level of complexity (as recited) that would render the process impossible/impractical for mental evaluation. See MPEP 2106.04(a)(2) subsection C. A Claim that Requires a Computer May Still Recite a Mental Process. Dependent claims are similarly analyzed at Prong One (e.g. claims 2-10) as they further comprise one or more limitations that may similarly be drawn under the mental processes Abstract Idea grouping (Step 2A, Prong One: Yes).
Step 2A Prong Two: This part of the eligibility analysis evaluates whether the claim integrates the recited judicial exception into a practical application (derived from Alice/Mayo step two and not to be conflated with an assessment of utility– MPEP 2103) of the exception. This evaluation is performed by (1) identifying whether there are any ‘additional elements’ recited in the claim beyond the judicial exception, and (2) evaluating those additional elements individually and in combination to determine whether the claim integrates the exception into a practical application. See MPEP 2106.04(d). Examiner notes for consideration at Prong Two of 2A that MPEP 2106.05(a), (b), (c), and (e) generally concern limitations that are indicative of integration, whereas 2106.05(f), (g), and (h) generally concern limitations that are not indicative of integration. As an additional note, ‘additional elements’ are generally limitations excluded from interpretation under the Abstract Idea groupings and may comprise portions of limitations otherwise identified as falling under those Abstract Idea groupings of the 2019 PEG (e.g. any ‘determination’ that may be made mentally accompanied by the use of generic computer hardware considered under the ‘apply it’ considerations of 2106.05(f)). Any ‘providing’/outputting broadly, and ‘collection’ of data (i.e. image acquisition(s)), be they images visually observable/ evaluated by a user/operator, also fail(s) to integrate at least in view of MPEP 2106.05(g) (extra-solution data gathering/output). The same determination holds for dependent claims that serve to limit the collection of data/images (by means of what is collected based on recited conditions) and/or introduce limitations generally linking to a field of use. None of the instant claims appear to explicitly/clearly capture/recite any disclosed improvement in technology (see MPEP 2106.05(a), with note that ‘functioning of a computer’ concerns functions integral to the way a computer operates and not ‘functions’ that a generic computer can be programmed/adapted to perform (see also 2106.05(f))) and any ‘additional elements’, even when considered in combination, fail to integrate at Prong Two of Step 2A accordingly. Integration in view of subsection (a) of 2106.05 requires an identification of the manner in which the improvement is achieved, to be explicitly and specifically (not at a high level of generality i.e. the ‘use of’ complementary modalities/data broadly for any conceivable satellite image analysis not inconsistent with diverse example embodiments) recited in the claims, as ‘additional elements’ precluded from interpretation under any of the Abstract Idea groupings (since the improvement cannot be to the exception itself).
Regarding claims 2 and 12, these claims fail to integrate the use of the recited ‘machine learning model (MLM)’ into a practical application. Although the use of MLMs are recited throughout the incident disclosure, the claims as recited fail to provide information about how the MLM operates to perform the recited method and/or system steps. As such, interpretation using the plain meaning of ‘identifying’ does not preclude a human user from conducting the same steps as the MLM, and therefore, does not sufficiently integrate the MLM into a practical application. See below for an excerpt from Example 47 of the 2024 PEG, as the provided interpretation of ‘ANN’ is similar to the incident use of ‘MLM’. Note that “selecting a particular data source or type of data to be manipulated” is an explicit example of an ‘insignificant extra-solution activity’ per MPEP 2106.05(g). Additionally, see below for MPEP 2106.05(a) regarding improvements to computer functioning, noting that the claims as recited can all be performed mentally without unreasonable time or effort.
Regarding claims 11-20, the recited ‘electronic device’, ‘non-transitory computer-readable storage medium’, and/or ‘one or more processors’ add only generic computer components to the proposed method and thus, do not integrate the recited exception into a practical application. Per Applicant’s specification at [0011], “In some implementations, an electronic device has a processor (e.g., one or more processors) that executes instructions stored in a non-transitory computer- readable medium to perform a method”. This instance is exemplary of how Applicant’s specification fails to establish how they specifically improve the present technology. Furthermore, the ‘electronic device’, ‘non-transitory computer-readable storage medium’, and/or ‘one or more processors’ in the claims as recited fail to integrate the exception into a practical application. See MPEP 2106.05(a) below for further clarification.
2024 PEG, Example 47:
The limitations “(a) receiving, at a computer, continuous training data” and “(f) outputting the anomaly data from the trained ANN” are mere data gathering and output recited at a high level of generality, and thus are insignificant extra-solution activity
MPEP 2106.05(a):
It is important to note that in order for a method claim to improve computer functionality, the broadest reasonable interpretation of the claim must be limited to computer implementation. That is, a claim whose entire scope can be performed mentally, cannot be said to improve computer technology.
Examples that the courts have indicated may not be sufficient to show an improvement in computer-functionality:
iii. Mere automation of manual processes, such as using a generic computer to process an application for financing a purchase, Credit Acceptance Corp. v. Westlake Services, 859 F.3d 1044, 1055, 123 USPQ2d 1100, 1108-09 (Fed. Cir. 2017) or speeding up a loan-application process by enabling borrowers to avoid physically going to or calling each lender and filling out a loan application, LendingTree, LLC v. Zillow, Inc., 656 Fed. App'x 991, 996-97 (Fed. Cir. 2016) (non-precedential)
It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981))
To show that the involvement of a computer assists in improving the technology, the claims must recite the details regarding how a computer aids the method, the extent to which the computer aids the method, or the significance of a computer to the performance of the method. Merely adding generic computer components to perform the method is not sufficient. Thus, the claim must include more than mere instructions to perform the method on a generic component or machinery to qualify as an improvement to an existing technology
Even when viewed in combination, the ‘additional elements’ present (namely the use of generic computer/technology components, insignificant extra-solution activity, and necessary data gathering and outputting) do not integrate the recited judicial exception into a practical application, the claims are directed to the judicial exception (Step 2A Prong Two: No).
Step 2B: This part of the eligibility analysis evaluates whether the claim amounts to ‘significantly more’ than the recited exception, i.e., whether any ‘additional element’, or combination of additional elements, adds an inventive concept to the claim. The considerations of Step 2A Prong 2 and Step 2B overlap but differ in that 2B also requires considering whether the claims/limitations feature any “specific limitation(s) other than what is well-understood, routine, conventional activity in the field” (WURC) (MPEP 2106.05(d)). Such a limitation if specifically recited however, must still be excluded from interpretation under any of the Abstract Idea groupings (else it is not an ‘additional element’ and extra-solution activity need not then be implicated). Limitations precluded from serving as indications of an inventive concept/ ‘significantly more’ include those that are not specifically recited (instead recited at a high level of generality), those that are established as WURC, those that are not ‘additional elements’ by nature of their analysis at Prong One (i.e. directed to the exception), and/or those that otherwise fail to serve for integration at Prong Two of 2A in view of 2106.05(f), (g) and/or (h). The claim(s) in question recite little beyond those limitations recited at a high level of generality and falling under the exception. (Step 2B: No).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 4, 7-9, 11, 12, 14, and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Gao (ICAN: Instance-Centric Attention Network for Human-Object Interaction Detection) in view of Lin (Microsoft COCO: Common Objects in Context).
Regarding claim 1, Gao teaches a method comprising: at an electronic device having a processor (“Using a single NVIDIA P100 GPU, our method (with late score fusion) takes less than 75ms to process” [page 8]): one or more frames depicting a living entity and one or more objects (see Figure 1 below, where the frame depicts a living entity (e.g. people) and one or more objects (e.g. cake, knife)); the event involving the living entity and a subset of the one or more objects (see Figure 1 below, where even though there are other objects making up the scene in the frame (e.g. flowers, silverware, etc.), only some objects (e.g. cake, knife) and the people in the frame make up the event); identifying the one or more objects depicted in the one or more frames (see Figure 1 below, where several objects are depicted and identified in the frame); and identifying an event (see Figure 5 below, where the ‘events’ are identified and clearly labeled as captions underneath the respective frames) based on the living entity and the one or more objects (see Figure 5 below, noting that the captions underneath each frame are the identified event based off of the living entity and objects within the colored bounding boxes). Gao falls silent regarding obtaining one or more frames of a video; identifying the subset of the one or more objects involved in the event; and segmenting the living entity and the subset of the one or more objects involved in the event in the one or more frames, wherein the segmenting comprises identifying portions (e.g., pixels) of the one or more frames corresponding to the living entity and the subset of the one or more objects.
Gao Figure 1
PNG
media_image1.png
449
299
media_image1.png
Greyscale
Gao Figure 5
PNG
media_image2.png
359
892
media_image2.png
Greyscale
Lin, however, teaches obtaining one or more frames of a video, the one or more frames depicting a living entity and one or more objects within a three-dimensional (3D) environment (“We introduce a large, richly-annotated dataset comprised of images depicting complex everyday scenes of common objects in their natural context” [page 1]. Examiner notes that ‘images’ of a ‘scene’ are comparable to Applicant’s specification at [0054], where “Some implementations identify the locations of the objects (associated with an event) in only a subset of the frames (e.g., an initial frame…)); identifying the subset of the one or more objects involved (see Fig. 11 and TABLE 2 below, which provide the 11 super-categories of detected objects and the list of all categories which make up the super-categories, respectively. Examiner notes additionally that for an alternate interpretation, TABLE 3 provides the list of all scene categories represented/available in the COCO dataset. Examiner notes that Fig. 3, also below, provides an ‘identifying’ of the subset, as represented by the category icons in (a) Examiner further notes that the recited ‘event’ is present in Gao and is rejected accordingly); and segmenting the living entity and the subset of the one or more objects involved in the event in the one or more frames, wherein the segmenting comprises identifying portions (e.g., pixels) of the one or more frames corresponding to the living entity and the subset of the one or more objects (Examiner notes that per Fig. 3 (see below), the ‘living entity’ (e.g. dog, represented by a dog icon located in the ‘Animal’ super-category of Fig. 11 (see below)) and ‘subset of one or more objects’ (e.g. bottles, represented by a bottle icon located in the ‘Kitchen’ super-category of Fig. 11 (see below)). Examiner additionally notes that a full list of categories in the COCO dataset are provided below for reference).
Lin TABLE 2
PNG
media_image3.png
396
1067
media_image3.png
Greyscale
Lin Fig. 11
PNG
media_image4.png
238
1043
media_image4.png
Greyscale
Lin Fig. 3
PNG
media_image5.png
318
1075
media_image5.png
Greyscale
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the human-object interaction detection network as taught by Gao to include the video frame segmentation method as taught by Lin, the proposed motivation being that allowing for segmentation of the objects as an alternative to using simple bounding box detection as used in Gao might allow for further robustness of the model, as pixel-level detection as characteristic of segmentation allows for improved detection, especially when expanding model applications. Examiner notes that the a subset of the COCO dataset disclosed by Lin is suggested by Gao on page 7, so no modification is needed to simply apply the dataset of Lin to the model by Gao.
Regarding claim 2, Gao in view of Lin teaches the method of claim 1. Gao further teaches wherein said identifying the one or more objects comprises using a machine learning model trained using a fixed taxonomy of objects (“We train our network for 300K iterations on the V-COCO trainval set” [Gao page 7], noting that the V-COCO trainval set is a subset of the COCO set as disclosed by Lin, and, as disclosed in Lin, is a fixed taxonomy, or “91 object types” [Lin abstract]. Examiner notes that per Applicant’s specification at [0048], “Some implementations may utilize fixed taxonomies of objects in training such that the training is only exposed to limited sets of objects”. Examiner notes that 91 objects is limited, and further that a ‘taxonomy’ can be clearly seen in Lin Fig.11 above, with the 91 categories sorted into 11 super-categories).
Regarding claim 4, Gao in view of Lin teaches the method of claim 1. Gao teaches further comprising: identifying an event type based on the segmented at least one of the one or more objects (see Gao Figure 1 above under claim 1 rejections, which demonstrates the ‘identifying’ of an event type based on an object of interest (Examiner notes that the object of interest (e.g. knife) is present in several of the listed event types: as both a ‘cutting instrument’ used by the person and as an object being ‘held’ by the person. Both of these types of events are detected by the model as human-object interactions). Examiner notes also that although Figure 5 relies on bounding boxes rather than segmentation as disclosed, Lin discloses the necessary segmentation (see Lin Fig. 3 above under claim 1 rejections)).
Regarding claim 7, Gao in view of Lin teaches the method of claim 1. Gao further teaches wherein said identifying the one or more objects and said identifying the event occur in parallel (see Figure 3 provided below, noting that the processing of the object configuration, the human configuration, and the layout between them are processed in parallel and are aggregated to determine a final output decision. Examiner further notes that per the rejection of claim 7 under 112(b), with reference to Applicant’s specification and the broadest reasonable interpretation of the recited language, some preliminary object detection must take place before the recited identifying of objects and events occur in parallel. In light of this interpretation, the parallel processing occurs between the determination of a ‘major event’ and the objects involved in such an event, which are determined in parallel as demonstrated in Figure 3, since the output of this processing is the human object interaction, or HOI, which defines which objects a person is interacting with and which ‘event’ the interaction should be labeled as).
Gao Figure 3
PNG
media_image6.png
393
890
media_image6.png
Greyscale
Regarding claim 8, Gao in view of Lin teaches the method of claim 1. Gao further teaches wherein said identifying the one or more objects and said identifying the event occur via execution of a single machine learning model (“The proposed model consists of following three major streams: (1) a human stream for detecting interaction based on human appearance; (2) an object steam that predicts the interaction based on object appearance; (3) a pairwise stream for encoding the spatial layouts between the human and object bounding boxes.” [page 4], Examiner notes that Gao only refers to ‘model’ and never ‘models’, so it is clear that their disclosure is performed on a single model).
Regarding claim 9, Gao in view of Lin teaches the method of claim 1. Lin further teaches wherein said identifying the subset of the one or more objects involved in the event comprises grouping the one or more objects to identify the subset (see Fig. 11 above under claim 1 rejections for object groupings which make up the super-category subsets, noting that this grouping is necessary for identification of the subsets based on detected objects. See Fig. 3 (a), also above under claim 1 rejections, for a visual representation of the involved object subset identification in light of the icons presented by Fig. 11).
Claim 11 is the electronic device claim corresponding to the method of claim 1 and is rejected accordingly. See [0051] and [0057] of Yadav regarding hardware required by the incident device claim.
Claim 12, dependent on 11, discloses the same embodiments as claim 2, dependent on claim 1, and is rejected accordingly. See claim 2 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Claim 14, dependent on 11, discloses the same embodiments as claim 4, dependent on claim 1, and is rejected accordingly. See claim 4 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Claim 17, dependent on 11, discloses the same embodiments as claim 7, dependent on claim 1, and is rejected accordingly. See claim 7 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Claim 18, dependent on 11, discloses the same embodiments as claim 8, dependent on claim 1, and is rejected accordingly. See claim 8 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Claim 19, dependent on 11, discloses the same embodiments as claim 9, dependent on claim 1, and is rejected accordingly. See claim 9 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Claim 20 is the non-transitory computer-readable storage medium claim corresponding to the method of claim 1 and is rejected accordingly. See [0057] of Yadav regarding hardware required by the incident CRM claim.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Gao (ICAN: Instance-Centric Attention Network for Human-Object Interaction Detection) in view of Lin (Microsoft COCO: Common Objects in Context) and further in view of Zheng (Open-Category Human-Object Interaction Pre-training via Language Modeling Framework).
Regarding claim 3, Gao in view of Lin teaches the method of claim 1. Both fall silent regarding wherein said identifying the one or more objects comprises enabling an open vocabulary event segmentation process comprising segmenting at least one of the one or more objects.
However, Zheng teaches wherein said identifying the one or more objects comprises enabling an open vocabulary event segmentation process comprising segmenting at least one of the one or more objects (“we reformulate HOI learning as a language sequence generation problem…which enables our model to leverage an open-set vocabulary, generating HOI triplets with a high degree of freedom” [page 19392], noting also that “our model improves its generalization to a wide range of novel HOIs” [page 19393]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the human-object interaction detection and segmentation method as taught by Gao in view of Lin to include open-category capabilities as taught by Zheng, the proposed motivation being that training a generalized model on a fixed library that is capable of accurately detecting and labeling a wide variety of events increases the usability and implementation possibilities of the proposed model.
Claim 13, dependent on 11, discloses the same embodiments as claim 3, dependent on claim 1, and is rejected accordingly. See claim 3 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Claims 5, 6, 10, 15, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Gao (ICAN: Instance-Centric Attention Network for Human-Object Interaction Detection) in view of Lin (Microsoft COCO: Common Objects in Context) and further in view of Xia (Human-object interaction recognition based on interactivity detection and multi-feature fusion).
Regarding claim 5, Gao in view of Lin teaches the method of claim 1. Both fall silent regarding wherein said identifying the one or more significant events is based on an event importance criteria.
Xia, however, teaches wherein said identifying the event comprises identifying one or more significant events associated with the one or more objects being depicted within the video (see Fig. 9, provided below, for several events occurring in the same frame with significance scores below them for clear visualization of which events are depicted as more or less important based on the described method).
Xia Fig. 9
PNG
media_image7.png
627
898
media_image7.png
Greyscale
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the human-object interaction detection and segmentation method as taught by Gao in view of Lin to include interactivity and classification scoring capabilities as taught by Xia, the proposed motivation being that implementing a significance metric to identify the ‘main’ event taking place in the frame would allow for a user to quickly identify the most important interactions, which would be helpful in, for example, surveillance implementations, where quickly detecting specific behaviors is important for effective use of the model.
Regarding claim 6, Gao in view of Lin and Xia teaches the method of claim 1. Xia further teaches wherein said identifying the one or more significant events is based on an event importance criteria (“Interactivity detection module aims at judging whether there is interaction between human–object pairs and calculating the interactivity score. According to the score, non-interactive pairs are filtered out. The remaining inter active pairs are transferred to the subsequent classification network” [page 1171], noting further that, “To extract more discriminative HOI classification features, we design the human–object pair-level contextual features and three component human pose features” [page 1173]. Examiner notes that removing non-interactive events and considering the body positioning/orientation of the person in the event when assigning an interaction classification to the event can be considered an ‘importance criteria’ under broadest reasonable interpretation, as it determines which objects are necessary for a given event to occur. Examiner further notes that Applicant provides no examples of the recited ‘importance criteria’. Per Applicant’s specification, an ‘event’ constitutes an interaction between at least one living entity and at least one object (see [0004]). Note further that detecting human pose features provides information to the model about the significance of a person’s orientation while interacting with an object. Note in Xia at [page 1174], “We observe that the importance of each body part varies when people interact with different objects. For example, when performing the behavior of using a laptop, the human hands and head are more important. In contrast, when performing the action of playing football, the human feet are more important.” Note that the above disclosure from Xia and Applicant’s specification evidences several ‘importance criteria’ under broadest reasonable interpretation).
Regarding claim 10, Gao in view of Lin teaches the method of claim 9. Xia further teaches wherein the subset of the one or more objects involved in the event comprise the most important objects involved in the event (“Interactivity detection module aims at judging whether there is interaction between human-object pairs and calculating the interactivity score. According to the score, non-interactive pairs are filtered out” [page 1171]. Examiner notes that in the context of the proposed model where an ‘event’ constitutes an interaction between at least one living entity and at least one object per Applicant’s specification at [0004], the ‘most important objects’ would be ones that are part of the event using the recited criteria for an ‘event’, meaning that they must be interacting with the living entity in the frame. Thus, filtering out detected objects that do not meet that criteria ensures that the subset of objects in the frame are the ‘most important’ ones for a given event).
Claim 15, dependent on 11, discloses the same embodiments as claim 5, dependent on claim 1, and is rejected accordingly. See claim 5 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Claim 16, dependent on 15, discloses the same embodiments as claim 6, dependent on claim 1, and is rejected accordingly. See claim 6 for limitation rejections and see claim 11 regarding disclosure containing necessary hardware to constitute a ‘device’.
Additional References
Prior art made of record and not relied upon that is considered pertinent to
applicant’s disclosure:
Additionally cited references (see attached PTO-892) otherwise not relied upon
above have been made of record in view of the manner in which they evidence the
general state of the art. Examiner notes that Afanasev (US20260170833A1) and Lee (20260112162A1), while not prior art to the incident application, suggest at least disclosure that serves to evidence the current state of the art of the disclosed inventive concept, and is noted for record.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JULIA T RAMETTA whose telephone number is (571)272-0451. The examiner can normally be reached Monday- Friday, 8 a.m. 5 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JULIA T RAMETTA/Examiner, Art Unit 2669 /CHAN S PARK/Supervisory Patent Examiner, Art Unit 2669
Prosecution Insights