DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed 06/08/2026 in response to the Non-Final Office Action mailed 01/05/2026 has been entered.
Claims 12-13 are currently pending and not withdrawn from consideration/examination, in U.S. Patent Application No. 18/386,051 and an Office action on the merits follows.
Terminal Disclaimer
The terminal disclaimer filed on 06/08/2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date(s) of US 11,854,215 and/or US 11,417,002 has been reviewed and is accepted. Double Patenting rejections to the claims are rendered moot/withdrawn accordingly. Examiner does note that while withdrawn claims are not being examined, there do appear to be similarities between one or more withdrawn claims of the instant application and those of DIV 19/396,935. Examiner does not predict this will cause any issues in the event that the considered claims are prosecuted to the point of being in condition for allowance, because those withdrawn claims may ultimately be cancelled in accordance with practices as described in e.g. MPEP §§ 821.02 & 1302.04 (see Form ¶ 8.07, and example/correction (D) for claims directed to a non-elected invention). If it is at all helpful to Applicant, Examiner understands the previous Restriction Requirement to have been properly made, and despite the current USPC assignment for the child/DIV application (which is subject to change), if consonant with the requirement and avoiding the elected subject matter for the instant application a child application directed to non-elected subject matter will likely be examined in an area other than USPC 382 - e.g. 348/218 or whichever area is most suited for examining sensor configurations that are novel for reasons related to their configuration independent from any subsequent processing that may be applied to data acquired therewith.
Response to 35 USC § 112 Rejections
In view of the foregoing amendments striking that instance of language ‘configuring’, claim rejections under 35 U.S.C. § 112(b) are withdrawn. The previously provided rejection (01/05/2026 Non-Final at page 3) did raise the issue of the manner in which the language “human detection for the object” may be ambiguous, for reasons distinct from those associated with the ‘configuring’, however in view of the accompanying amendment (which now requires that contours associated with the object be separated/separatable into head portion, torso portion, and limb portion) interpretation is taken such that instances/language regarding ‘human’ and ‘object’ reference the same imaged entity.
Response to Arguments/Remarks
Applicant's arguments filed 06/08/2026, asserting that the prior combination of references fails to fairly teach/suggest contour separation/processing, arrangement analysis, and state determination(s) therefrom, have been fully considered and determined persuasive. Updated search and consideration in view of the claim(s) as amended has provided at least those recently cited references, evidencing the obvious nature of the claims as now amended and the new ground(s) of rejection rendered below.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. Applicant has not complied with one or more conditions for receiving the benefit of at least one earlier filing date under 35 U.S.C. 120 (in view of Application No. 15/361,378) as follows:
The later-filed application must be an application for a patent for an invention which is also disclosed in the prior application (the parent or original nonprovisional application or provisional application). The disclosure of the invention in the parent application and in the later-filed application must be sufficient to comply with the requirements of 35 U.S.C. 112(a) or the first paragraph of pre-AIA 35 U.S.C. 112, except for the best mode requirement. See Transco Products, Inc. v. Performance Contracting, Inc., 38 F.3d 551, 32 USPQ2d 1077 (Fed. Cir. 1994).
The disclosure of at least one prior-filed application, Application No. 15/361,378, fails to provide adequate support or enablement in the manner provided by 35 U.S.C. 112(a) or pre-AIA 35 U.S.C. 112, first paragraph for claims 12-13 of this application. More specifically the ‘378 Application fails to meet written description requirements for those claim limitations concerning Figs. 4-6, S120, S130 and S140 of the instant application. Accordingly, claims 12-13 are being examined as entitled to a date/benefit no earlier than the 11/26/2016 Filing Date of 15/707,583 (CIP of the referenced ‘378 Application) (consistent with the 01/05/2026 SRNT).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Batchuluun et al. “Robust Behavior Recognition in Intelligent Surveillance Environments” (30 June 2016), in view of Yao et al. “Animated Pose Templates for Modeling and Detecting Human Actions” (2014).
As to claim(s) 12-13, Batchuluun discloses a method for image recognition (page 1 Abs “These systems should be operated in both daytime and nighttime, but objects are invisible in images captured by visible light camera during the night. Therefore, near infrared (NIR) cameras, thermal cameras (based on medium-wavelength infrared (MWIR), and long-wavelength infrared (LWIR) light) have been considered for usage during the nighttime as an alternative … a dual system of visible light and thermal cameras is used in our research, and we propose a new behavior recognition in intelligent surveillance environments”, etc.,) comprising:
detecting whether an object is detected (page 4 Fig. 1, § 2.1, producing that fused bounding box/detection result, based on corresponding human regions detected from the visible light and thermal images, prior to detected region analysis for recognizing human behavior);
capturing at least one image by a sensor module (page 3 section 2 “We have implemented dual camera systems (including an FLIR thermal camera capturing the images of 640 × 480 pixels [44] and a visible light camera capturing the images of 800 × 600 pixels) … Therefore, the visible light and thermal images are simultaneously captured with minimum disparity”, page 4 Fig. 1);
performing, by the sensor module (page 3 § 2 initial human detection and subsequent behavior recognition based thereon, in view interpretation in which the recited object and human are the same entity, given the ‘separating’ below, § 2.1 “In this section, we present a brief introduction of our human detection system because the behavior recognition is carried out based on the detection results”, etc.,), object recognition and human detection for the object, wherein the sensor module includes a plurality of first sensors and a plurality of second sensors, the first sensors are image sensors for the object recognition and the second sensors are thermal sensors for the human detection (page 1 Abs, page 2 §1 “A thermal camera image clearly reveals the human body at night and in winter by measuring the human body temperature, which is detectable in the range of MWIR of 3–8 μm and LWIR of 8–15 μm [1–12]. … However, the distinction between the human and the background in the thermal image diminishes when the background temperature is similar to that of the human in some situations during the daytime, which can reduce the consequent recognition accuracy of behavior. Therefore, the use of both thermal and visible light images enables us to enhance the recognition accuracy of behavior in intelligent surveillance system”, page 4 “The use of dual camera systems enables our method to robustly detect human areas in the images of various environments such as those with severe shadow, illumination variations, darkness, and cases in which the background temperature is similar to human area in hot weather”, etc.,);
wherein the performing of the object recognition and the human detection (§ 2, 2.1 disclosing initial box detection, page 5 § 2.2.1 human behavior recognition) includes:
analyzing arrangements ofpage 5 § 2.2.1 “The method proposed for behavior recognition is presented in Figure 3. Detailed explanations of rules 1-4 are provided in Section 2.2.2. We adopted an IF-THEN rule-based method to recognize behavior. As shown in Figure 3, behaviors are roughly divided into two types according to the horizontal activeness measured by the change in the width of the detected human box”, page 5 Fig. 3; Examiner notes Batchuluun evidences the obvious nature of behavior recognition on the basis of arrangements of the detected contour broadly (even if not separated into head, torso, and limb portions), given those determined temporal and spatial changes to the arrangement, and comparison to reference patterns (Fig. 4), i.e. a change of center position of the box in the vertical direction, dimension changes to the box, and associated movement speeds in either/both instances, Fig. 8, etc.,); and
determining a state of the object based on the arrangements of Figures 3-4, page 6 Figure 4 “Comparison of motion patterns for each type of behavior”, page 6 § 2.2.1 “In our research, the 11 types of behavior are categorized into three classes, with each class representing behavior with different intention and meaning”, Class 1: Walking, running, sitting and standing § 2.2.2 at page 7, Class 2 Kicking, punching, lying down, waving with two hand, and waving with one hand § 2.2.3 at pages 8-11, Class 3 Approaching and leaving, Fig. 8 illustrating distinct limb portions and associated arrangements, etc., see also Fig. 13 at page 16 of 23; Examiner also notes that while the compared methods including Fourier Descriptor-based and GEI-based methods are described in the context of a comparison to background art, relevant disclosure at pages 17-19 further illustrates the obvious nature of and/or state of the art as it relates to determining object/human state based on detected contours broadly),
wherein, when the first sensors and the second sensors are arranged… (page 4 Figure 1, illustrating visible and thermal sensor portions disposed in a common plane, see also pages 8-9 of the Non-Final Office Action mailed 01/05/2026, describing with reference to MPEP 2111.04, how contingent/conditional limitations (also presented in the alternative) may result in none assigned patentable weight particularly if conditional requirements are or need not be met (e.g. conditional language ‘when’), and the manner in which ‘wherein’ clauses, in a case-specific manner, may give rise to questions regarding limiting effect and may not be considered material to patentability (for the elected process) if not integral to the implementation of any specifically recited steps of the method/process. In other words, consonant with the restriction requirement and Applicant’s election of the subject matter of Fig. 6, sensor configuration specifics that are structural in nature concern non-elected subject matter).
Batchuluun fails to explicitly disclose separating a contour of the object retrieved from the at least one image into a plurality of portions, wherein the plurality of portions include a head portion, a torso portion, and limb portions; and arrangement(s) based on all of those separated portions as recited. Batchulun at best suggests separating a contour into e.g. two distinct limb portions (Fig. 8).
PNG
media_image1.png
450
1094
media_image1.png
Greyscale
Yao however evidences the obvious nature of separating a contour of an object retrieved from at least one image into a plurality of portions, wherein the plurality of portions include a head portion, a torso portion, and limb portions (page 4 Fig. 4 see yellow head portion, red torso/chest portion, blue arm, and green legs; see also page 5 Fig. 5 illustrating shape templates that comprise those parts/bounding boxes for head, torso, limbs, etc., at leaf nodes (dependent upon how much of the object is present/captured in the frame), Fig. 6, Fig. 9, Fig. 10, etc.,); and an arrangement analysis therebetween for the purposes of action/behavior/state recognition (page 1 § 1.1 “Building a robust system for real-world human action understanding presents challenges at multiple levels: 1) localizing the actions of interest; 2) recognizing the actions; and 3) interpreting the interactions between agents and contextual objects”, page 2 “If we compare with the task of object recognition, it is worth noting that a quantum jump in performance has been achieved in the past few years when researchers departed from the BoW features and adopted the deformable part-based model (DPM) [10], especially for human detection in images [11], [12]. Thus we expect that a better representation for human action should extend the DPM to the temporal domain and capture some important information missed by the STIP representations”, page 3 § 2.1 “Each action is composed of a sequence of key poses or action snippets, and the number of poses depends on the complexity of the action”, page 7 § 2.3, pages 8-9 § 3 Inference, Fig. 10, etc.,).
PNG
media_image2.png
655
746
media_image2.png
Greyscale
PNG
media_image3.png
968
920
media_image3.png
Greyscale
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the behavior/action analysis processing of Batchuluun to further comprise separating a contour of an object retrieved from at least one image into a plurality of portions, wherein the plurality of portions include a head portion, a torso portion, and limb portions as taught/suggested by Yao (and also in Batchuluun but for limb portions without necessarily head and torso portions), the motivation as similarly taught/suggested therein and readily recognized by POSITA that such a separation/ decomposition would enable a deformable parts analysis (e.g. one that may assume the parts are ‘independent’ of each other given a root template location) better suited for detecting a wider array of actions/behavior (e.g. using vending machine comprising more nuanced actions including approaching machine, pushing button, and picking up object).
Additional References
Prior art made of record and not relied upon that is considered pertinent to applicant's disclosure:
Additionally cited references (see attached PTO-892) otherwise not relied upon above have been made of record in view of the manner in which they evidence the general state of the art.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IAN L LEMIEUX whose telephone number is (571)270-5796. The examiner can normally be reached Mon - Fri 9:00 - 6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached on 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IAN L LEMIEUX/Primary Examiner, Art Unit 2669
Prosecution Insights