Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of Applicant’s claim of the present application claiming priority and benefit under 35 U.S.C. 119(a-d) to Taiwanese Patent Application No. TW-113117800 filed 05/14/2024.
Information Disclosure Statement
The information disclosure statements (“IDS”) filed 11/25/2024 and 11/17/2025 have been reviewed and the listed references were noted.
Drawings
The 12-page drawings have been considered and placed in the file.
Status of Claims
Claims 1-20 are pending.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a person detection module…” and “an onlooker determination module…” in claims 1 and 11, “a neural network module…” in claims 9 and 19, and “an output feature generation module…” and “prediction modules…” in claims 10 and 20.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter (an abstract idea without significantly more). The claims recite a system and method for detecting onlookers. With respect to analysis of claim 1:
Step 1:
With regard to Step 1, the instant claim is directed to a system; and therefore, the claim is directed to one of the statutory categories of inventions.
Step 2A, Prong One:
With regard to Step 2A, Prong One, the limitations of “a person detection module, configured to receive an image, and obtain, in response to presence of at least one person in the image, person information of each of the at least one person, wherein the person information comprises distance information relative to a device; and an onlooker determination module, configured to: (a) determine whether the at least one person comprises at least one non-user present in a range based on the distance information of the person information of each of the at least one person; and (b) determine, in response to presence of the at least one non-user in the range, a security classification to which each of the at least one non-user belongs based on the person information of each of the at least one non-user, wherein the security classification comprises an onlooker category”, as drafted, recite an abstract idea, such as the combination of a system and using judgement and mathematical calculations that, under their broadest reasonable interpretation, covers the performance of the limitations manually of in the mind. That is, someone may determine if a person in an image is too close to the user and classify them based on that determination. This is the concept that falls under the combination of the groupings of abstract ideas of mental processes and judgement, i.e., a concept performed in the human mind, evaluation, judgement, and/or opinion.
Step 2A, Prong Two:
The 2019 PEG defines the phrase “integration into a practical application” to require an
additional element or a combination of additional elements in the claim to apply, rely on, or use
the judicial exception. In the instant case, there are no additional elements/limitations in the
claims, with the exception of the insignificant extra-solution activities, such as obtaining
image and distance data as input data. This limitation is regarded as insignificant extra-
solution activity of acquiring medical image data, i.e., gathering input information, which may
not be considered as an indication of integration of the judicial exception into a practical
application. Accordingly, the above-mentioned additional element/limitation does not
integrate the abstract idea into a practical application; and therefore, the claim recites an
abstract idea.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B.
The claims herein do not include additional elements that are sufficient to amount to
significantly more than the judicial exception, because as discussed above with respect to
integration of the abstract idea into practical application, the additional elements/limitations,
amount to no more than insignificant routine and conventional elements. The abstract
operation/purpose of the apparatus as listed in Claim 1, cannot provide an inventive concept.
Therefore, independent Claim 1 is not patent eligible.
Furthermore, with regard to dependent claims 2-10, viewed individually, these additional elements/limitations, under their broadest reasonable interpretation, are regarded as either providing additional abstract idea (i.e., judgement/evaluation or performing the limitations in the mind) or adding limitations that may not be considered as significantly more than the abstract idea; which do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Accordingly, they are not patent eligible.
Independent claim 11 is evaluated in the same manner as claim 1 under 35 U.S.C. Using the same evaluation as claim 1 for claim 11, claim 11 is not patent eligible. Furthermore, with regard to dependent claims 2-10 and 12-20, viewed individually, these additional elements/limitations, under their broadest reasonable interpretation, are regarded as either providing additional abstract idea (i.e., judgement/evaluation or performing the limitations in the mind) or adding limitations that may not be considered as significantly more than the abstract idea; which do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Accordingly, they are not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 8, 11, and 18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by David Coulon (US 2013/0021240 A1).
Regarding claim 1, Coulon teaches, “An onlooker detection system, comprising: a person detection module, configured to receive an image, and obtain, in response to presence of at least one person in the image, person information of each of the at least one person, wherein the person information comprises distance information relative to a device;” (Coulon, Abstract discloses; “A method for controlling an electronic apparatus, includes steps of: acquiring an image of the environment of the apparatus, detecting the presence of human faces in the image acquired, estimating a respective position of each face detected in relation to the apparatus”) “and an onlooker determination module, configured to: (a) determine whether the at least one person comprises at least one non-user present in a range based on the distance information of the person information of each of the at least one person;” (Coulon, Para. [0014] discloses; “According to one embodiment, the method comprises steps of determining whether or not each face detected in a field of view of the apparatus, is that of a user of the apparatus or of a person authorized by the user, or that of a potential intruder, based on the estimated distance between the apparatus and the detected face.”) “and (b) determine, in response to presence of the at least one non-user in the range, a security classification to which each of the at least one non-user belongs based on the person information of each of the at least one non-user, wherein the security classification comprises an onlooker category.” (Coulon, Para. [0014] discloses; “According to one embodiment, the method comprises steps of determining whether or not each face detected in a field of view of the apparatus, is that of a user of the apparatus or of a person authorized by the user, or that of a potential intruder, based on the estimated distance between the apparatus and the detected face.” Examiner interprets the apparatus of Coulon to put the non-users into an “intruder” category based on the person information.)
Regarding claim 8, Coulon teaches, “The onlooker detection system according to claim 1, wherein the onlooker determination module is configured to transmit, in response to determining that one of the at least one non-user belonging to the onlooker category, a signal to cause the device to start initiating an anti-peeping program.” (Coulon, Para. [0024] discloses; “In an embodiment, the protection function comprises at least one of the following operations: sending or displaying an alert signal; displaying the face of a person detected as an intruder; and enabling a privacy filter on a display screen of the apparatus.” Intruder is interpreted to be analogous to onlooker. The protection function (“anti-peeping program”) displays the face of the intruder, so it is implicit that the protection function (“anti-peeping program”) starts in response to a user being an intruder (“onlooker”).)
Claim 11 recites a method with steps corresponding to the elements of the system recited in Claim 1. Therefore, the recited steps of this claim are mapped to the proposed prior art reference in the same manner as the corresponding elements in its corresponding system claim. Thus, claim 11 is rejected in the same manner as claim 1.
Claim 18 recites a method with steps corresponding to the elements of the system recited in Claim 8. Therefore, the recited steps of this claim are mapped to the proposed prior art reference in the same manner as the corresponding elements in its corresponding system claim. Thus, claim 18 is rejected in the same manner as claim 8.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over David Coulon (US 2013/0021240 A1), in view of Huang et al. (CN 117574463 A).
Regarding claim 2, Coulon does not explicitly teach, “The onlooker detection system according to claim 1, wherein the security classification comprises a non-onlooker category, and step (b) comprises: (b1) determining, for a current object of the at least one non-user, whether the current object faces the device; determining that the current object belongs to the onlooker category in response to the current object facing the device; and determining that the current object belongs to the non-onlooker category in response to the current object not facing the device; and (b2) selecting, in response to the at least one non-user comprising an unselected object, an unselected one of the at least one non-user as the current object, and returning to step (b1).” Since Coulon does not explicitly disclose these limitations, Examiner relies on the teachings of Huang in an analogous field of endeavor. Specifically, Huang teaches, “The onlooker detection system according to claim 1, wherein the security classification comprises a non-onlooker category, and step (b) comprises: (b1) determining, for a current object of the at least one non-user, whether the current object faces the device;” (Huang, Abstract discloses; “The method comprises: under the condition that the terminal is in the interface protection mode, according to the human face information collected by the front image collecting device of the terminal, determining whether there is a face of the target object facing the terminal interface” It would be obvious to combine this disclosure of Huang with the categories of Coulon in the rejection of claim 1 to obtain this limitation and determine if the non-user is facing the device.) “determining that the current object belongs to the onlooker category in response to the current object facing the device; and determining that the current object belongs to the non-onlooker category in response to the current object not facing the device;” (Huang, Abstract discloses; “The method comprises: under the condition that the terminal is in the interface protection mode, according to the human face information collected by the front image collecting device of the terminal, determining whether there is a face of the target object facing the terminal interface” From Huang’s disclosure, it is implicit that Huang’s apparatus can determine if a person is not facing the device if it can determine if a person is facing the device. Additionally, it would be obvious to combine this disclosure of Huang with the categories of Coulon in the rejection of claim 1 to obtain this limitation and determine if the non-user is not facing the device.) “and (b2) selecting, in response to the at least one non-user comprising an unselected object, an unselected one of the at least one non-user as the current object, and returning to step (b1).” (Coulon, Para. [0010] discloses; “According to one embodiment, the method comprises steps of: estimating a respective position of each face detected in relation to the apparatus, and sending a signal to the apparatus to enable a function of the apparatus, if a condition is met relating to a number of faces detected in the image and/or the estimated position of each detected face.” The device of Coulon can detect multiple faces, and thus it would be obvious to perform steps b1 and b2 for each face iteratively.)
Coulon and Huang are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Coulon to incorporate the teachings of Huang in order to determine if a non-user is facing a camera. One would have been motivated to combine the previously described device of Coulon with the teachings of Huang to differentiate between people looking at device contents and not looking at the contents. Accordingly, it would have been obvious to combine Coulon and Huang to obtain claim 2.
Claim 12 recites a method with steps corresponding to the elements of the system recited in Claim 2. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Coulon and Huang references, presented in rejection of Claim 2, apply to this claim.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over David Coulon (US 2013/0021240 A1), in view of Shoda et al. (US 20260080395 A1 w/ EFD of 06/05/2023).
Regarding claim 3, Coulon does not explicitly teach, “The onlooker detection system according to claim 1, wherein the security classification comprises a passerby category and a sharing user category, and step (b) comprises: (b1) determining, for a current object of the at least one non-user, whether a distance between the current object and a user is less than a preset distance; determining, in response to the distance between the current object and the user being less than the preset distance, that the current object belongs to the sharing user category; determining, in response to the distance between the current object and the user being not less than the preset distance, whether the current object faces the device; determining that the current object belongs to the onlooker category in response to the current object facing the device, and determining that the current object belongs to the passerby category in response to the current object not facing the device; and (b2) selecting, in response to the at least one non-user comprising an unselected object, an unselected one of the at least one non-user as the current object, and returning to step (b1).” Since Coulon does not explicitly disclose these limitations, Examiner relies on the teachings of Shoda in an analogous field of endeavor. Specifically, Shoda teaches, “The onlooker detection system according to claim 1, wherein the security classification comprises a passerby category and a sharing user category,” (Shoda, Para. [0088] discloses; “Accordingly, the payment terminal 100 can determine presence or absence of peeping based on a direction in which the third party H2 is actually looking, thereby improving the determination accuracy for the peeping” It is implicit that the third party would be put into either a “peeping” (onlooker) or “non-peeping” (passerby) category based on if they are facing/looking at the device.) “and step (b) comprises: (b1) determining, for a current object of the at least one non-user, whether a distance between the current object and a user is less than a preset distance;” (Shoda, Para. [0077] discloses; “When the distance is small, there is a high possibility that the third party H2 is the peeper. Therefore, for example, when the estimated distance is equal to or less than a predetermined threshold th1 (predetermined distance), the second CPU 21 determines that the third party H2 is the peeper.”) “determining, in response to the distance between the current object and the user being less than the preset distance, that the current object belongs to the sharing user category;” (Shoda, Para. [0077] discloses; “When the distance is small, there is a high possibility that the third party H2 is the peeper. Therefore, for example, when the estimated distance is equal to or less than a predetermined threshold th1 (predetermined distance), the second CPU 21 determines that the third party H2 is the peeper.” Examiner interprets that when the distance is less than a threshold, Shoda is implying that the object belongs to a category. It is merely a design choice to name the category “peeper” or “sharing user”.) “determining, in response to the distance between the current object and the user being not less than the preset distance, whether the current object faces the device;” (Shoda, Para. [0077] discloses; “When the distance is small, there is a high possibility that the third party H2 is the peeper. Therefore, for example, when the estimated distance is equal to or less than a predetermined threshold th1 (predetermined distance), the second CPU 21 determines that the third party H2 is the peeper.” And Shoda, Para. [0088] discloses; “Accordingly, the payment terminal 100 can determine presence or absence of peeping based on a direction in which the third party H2 is actually looking, thereby improving the determination accuracy for the peeping” Shoda teaches determining a distance between objects based on a threshold and determining if the object is facing/looking at a device. Thus, Shoda has the capabilities to perform this limitation and it would be obvious to combine these disclosures to obtain this limitation.) “determining that the current object belongs to the onlooker category in response to the current object facing the device, and determining that the current object belongs to the passerby category in response to the current object not facing the device;” (Shoda, Para. [0088] discloses; “Accordingly, the payment terminal 100 can determine presence or absence of peeping based on a direction in which the third party H2 is actually looking, thereby improving the determination accuracy for the peeping” It is implicit that the third party would be put into either a “peeping” (onlooker) or “non-peeping” (passerby) category based on if they are facing/looking at the device.) “and (b2) selecting, in response to the at least one non-user comprising an unselected object, an unselected one of the at least one non-user as the current object, and returning to step (b1).” (Coulon, Para. [0010] discloses; “According to one embodiment, the method comprises steps of: estimating a respective position of each face detected in relation to the apparatus, and sending a signal to the apparatus to enable a function of the apparatus, if a condition is met relating to a number of faces detected in the image and/or the estimated position of each detected face.” The device of Coulon can detect multiple faces, and thus it would be obvious to perform steps b1 and b2 for each face iteratively.)
Coulon and Shoda are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Coulon to incorporate the teachings of Shoda in order to determine categories for a non-user based on relative distance. One would have been motivated to combine the previously described device of Coulon with the teachings of Shoda to differentiate between people looking at device contents and not looking at the contents. Accordingly, it would have been obvious to combine Coulon and Shoda to obtain claim 3.
Claim 13 recites a method with steps corresponding to the elements of the system recited in Claim 3. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Coulon and Shoda references, presented in rejection of Claim 3, apply to this claim.
Claims 4, 5, 14, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over David Coulon (US 2013/0021240 A1), in view of Huang et al. (CN 117574463 A), in further view of Sivagnanenthirarajah et al. (US 2023/0359468 A1), and still in view of Harada et al. (US 20230023251 A1).
Regarding claim 4, the combination of Coulon and Huang teaches, “The onlooker detection system according to claim 2, wherein the person information comprises face information, (Coulon, Abstract discloses; “A method for controlling an electronic apparatus, includes steps of: acquiring an image of the environment of the apparatus, detecting the presence of human faces in the image acquired, estimating a respective position of each face detected in relation to the apparatus” The “presence of human faces” is interpreted to be face information.) “and the step of determining whether the current object faces the device comprises: (b11) determining, based on face information of the current object, whether a face of the current object is detected;” (Coulon, Abstract discloses; “A method for controlling an electronic apparatus, includes steps of: acquiring an image of the environment of the apparatus, detecting the presence of human faces in the image acquired” It is implied in this disclosure by Coulon that detecting a face is based on the presence of human faces. It is also implied that the apparatus could not detect a face if there is no human presence.) corresponding to the face information in response to the face of the current object being detected;” (Sivagnanenthirarajah, Claim 18 discloses; “The method of claim 17, wherein determining that the face is oriented towards the display device is based on determining that a facial yaw angle of the face is less than +/−45° relative to a perpendicular of the display device.” It would be obvious to include this angle information in the person information of Coulon.)
Coulon, Huang, and Sivagnanenthirarajah are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Coulon and Huang to incorporate the teachings of Sivagnanenthirarajah in order to use angle information to determine if a user is facing a device. One would have been motivated to combine the previously described device of Coulon and Huang with the teachings of Sivagnanenthirarajah to ensure that a user’s head is facing a device. Accordingly, it would have been obvious to combine Coulon, Huang, and Sivagnanenthirarajah to obtain the above specified limitations.
Further regarding claim 4, the combination of Coulon, Huang, and Sivagnanenthirarajah does not explicitly teach, “and (b13) determining, in response to the face of the current object not being detected, whether the current object faces the device based on key point information of the current object.” Since the combination of Coulon, Huang, and Sivagnanenthirarajah does not explicitly disclose this limitation, Examiner relies on the teachings of Harada in an analogous field of endeavor. Specifically, Harada teaches, “and (b13) determining, in response to the face of the current object not being detected, whether the current object faces the device based on key point information of the current object.” (Harada, Para. [0112] discloses; “Subsequently, the processor 122 estimates the presence or absence of the twist in the body of the person captured in the image I based on a ratio between the distance D1 and the distance D2. Specifically, when a difference between the ratio and 1 exceeds a threshold value, it is estimated that the body is twisted. When a person as the subject 30 faces the imaging device 11, it is highly probable that the left shoulder feature point LU1 and the right shoulder feature point RU1 are located symmetrically with respect to the face feature point F in the left-right direction (X direction).” It would be obvious to include this key point information in the person information of Coulon.)
Coulon, Huang, Sivagnanenthirarajah, and Harada are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Coulon, Huang, and Sivagnanenthirarajah to incorporate the teachings of Harada in order to use key point information to determine if a user is facing a device. One would have been motivated to combine the previously described device of Coulon, Huang, and Sivagnanenthirarajah with the teachings of Harada to ensure that a user’s head/body is facing a device. Accordingly, it would have been obvious to combine Coulon, Huang, Sivagnanenthirarajah, and Harada to obtain claim 4.
Regarding claim 5, the combination of Coulon, Huang, Sivagnanenthirarajah, and Harada teaches, “The onlooker detection system according to claim 4, wherein the angle information of the current object corresponding to the face information comprises a head yaw angle, and step (b12) comprises: determining that the current object faces the device in response to the head yaw angle being in an angle threshold range;” (Sivagnanenthirarajah, Para. [0075] discloses; “For example, the face is determined to be oriented towards the display device based on determining that a facial yaw angle is less than +/−45° relative to the perpendicular of the display device.”) “and determining that the current object does not face the device in response to the head yaw angle being not in the angle threshold range.” (Sivagnanenthirarajah, Para. [0066] discloses; “For example, the face is determined to be not oriented towards the display device based on determining that a facial yaw angle is greater than +/−45° relative to the perpendicular of the display device.”) The proposed combination as well as the motivation for combining the Coulon, Huang, Sivagnanenthirarajah, and Harada references presented in the rejection of claim 4, apply to claim 5 and are incorporated herein by reference. Thus, the system recited in claim 5 is met by Coulon, Huang, Sivagnanenthirarajah, and Harada.
Claim 14 recites a method with steps corresponding to the elements of the system recited in Claim 4. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Coulon, Huang, Sivagnanenthirarajah, and Harada references, presented in rejection of Claim 4, apply to this claim.
Claim 15 recites a method with steps corresponding to the elements of the system recited in Claim 5. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Coulon, Huang, Sivagnanenthirarajah, and Harada references, presented in rejection of Claim 5, apply to this claim.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over David Coulon (US 2013/0021240 A1), in view of Huang et al. (CN 117574463 A), in further view of Sivagnanenthirarajah et al. (US 2023/0359468 A1), and still in view of Harada et al. (US 20230023251 A1), and finally in view of Molin et al. (US 20220410908 A1).
Regarding claim 6, the combination of Coulon, Huang, Sivagnanenthirarajah, and Harada does not explicitly teach, “The onlooker detection system according to claim 4, wherein the angle information of the face information of the current object comprises a gaze point yaw angle, and step (b12) comprises: determining that the current object faces the device in response to the gaze point yaw angle being in an angle threshold range; and determining that the current object does not face the device in response to the gaze point yaw angle being not in the angle threshold range.” Since the combination of Coulon, Huang, Sivagnanenthirarajah, and Harada does not explicitly disclose these limitations, Examiner relies on the teachings of Molin in an analogous field of endeavor. Specifically, Molin teaches, “The onlooker detection system according to claim 4, wherein the angle information of the face information of the current object comprises a gaze point yaw angle,” (Molin, Para. [0076] discloses; “Returning to FIG. 4A, for example, the rate at which the head yaw exceeds the rotation threshold can be calculated and compared to the gaze rate thresholds, so as to determine whether the diverted gaze event has occurred.” Examiner interprets this disclosure to define a diverted gaze based on “gaze point yaw angle”. It would be obvious to include the gaze point yaw angle in the angle information of Sivagnanenthirarajah) “and step (b12) comprises: determining that the current object faces the device in response to the gaze point yaw angle being in an angle threshold range;” (Molin, Para. [0074] discloses; “FIG. 4A shows, for example, a time history of driver head pose data characterizing the head yaw over time. The 90 degree mark may correspond to a full turn towards the passenger. The threshold mark (Thr) may indicate the rotation threshold past which the driver's gaze is considered to be diverted. The time periods for which the yaw exceeds the rotation threshold can be calculated and compared to the gaze held thresholds, so as to determine whether the diverted gaze event has occurred. For example, curves 402 and 404 may not trigger the diverted gaze event, due to not lasting long enough to exceed the gaze held threshold, whereas curve 406 may trigger the diverted gaze event, due to lasting sufficiently long enough to exceed the gaze held threshold.” “Diverted gaze” is considered analogous to not facing a device. Thus, Molin uses a gaze point yaw angle threshold to determine if a person is facing a device (non-diverted gaze).) “and determining that the current object does not face the device in response to the gaze point yaw angle being not in the angle threshold range.” (Molin, Para. [0074] discloses; “FIG. 4A shows, for example, a time history of driver head pose data characterizing the head yaw over time. The 90 degree mark may correspond to a full turn towards the passenger. The threshold mark (Thr) may indicate the rotation threshold past which the driver's gaze is considered to be diverted. The time periods for which the yaw exceeds the rotation threshold can be calculated and compared to the gaze held thresholds, so as to determine whether the diverted gaze event has occurred. For example, curves 402 and 404 may not trigger the diverted gaze event, due to not lasting long enough to exceed the gaze held threshold, whereas curve 406 may trigger the diverted gaze event, due to lasting sufficiently long enough to exceed the gaze held threshold.” “Diverted gaze” is considered analogous to not facing a device. Thus, Molin uses a gaze point yaw angle threshold to determine if a person is not facing a device (diverted gaze).)
Coulon, Huang, Sivagnanenthirarajah, Harada, and Molin are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Coulon, Huang, Sivagnanenthirarajah, and Harada to incorporate the teachings of Molin in order to use gaze point yaw angle information to determine if a user is facing a device. One would have been motivated to combine the previously described device of Coulon, Huang, Sivagnanenthirarajah, and Harada with the teachings of Molin to ensure that a user’s head/body is facing a device. Accordingly, it would have been obvious to combine Coulon, Huang, Sivagnanenthirarajah, Harada, and Molin to obtain claim 6.
Claim 16 recites a method with steps corresponding to the elements of the system recited in Claim 6. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Coulon, Huang, Sivagnanenthirarajah, Harada, and Molin references, presented in rejection of Claim 6, apply to this claim.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over David Coulon (US 2013/0021240 A1), in view of Balavalikar Krishnamurthy et al. (US 20240135748 A1).
Regarding claim 9, Coulon does not explicitly teach, “The onlooker detection system according to claim 1, wherein the person detection module comprises a neural network module, the neural network module is configured to receive the image, and output a plurality of information tensors in response to the presence of the at least one person in the image, and the person detection module is configured to output the person information of each of the at least one person based on the information tensors in response to the presence of the at least one person in the image.” Since Coulon does not explicitly disclose these limitations, Examiner relies on the teachings of Balavalikar Krishnamurthy in an analogous field of endeavor. Specifically, Balavalikar Krishnamurthy teaches, “The onlooker detection system according to claim 1, wherein the person detection module comprises a neural network module, the neural network module is configured to receive the image, and output a plurality of information tensors in response to the presence of the at least one person in the image, and the person detection module is configured to output the person information of each of the at least one person based on the information tensors in response to the presence of the at least one person in the image.” (Balavalikar Krishnamurthy, Para. [0049] discloses; “The disclosed methodology also detects, from each input frame image, one or more human heads for any meeting participants captured in the input frame image by applying a machine learning human head detector model to said input image frame. In selected embodiments, the human heads are detected by applying image pre-processing to each input frame image, applying a machine learning human head detector model to each input image frame to generate an output tensor for each detected human head and applying image post-processing to convert each output tensor to a head bounding box which surrounds a corresponding detected human head.” Examiner interprets a bounding box to be person information.)
Coulon and Balavalikar Krishnamurthy are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Coulon to incorporate the teachings of Balavalikar Krishnamurthy in order to determine person information using a neural network. One would have been motivated to combine the previously described device of Coulon with the teachings of Balavalikar Krishnamurthy to apply a neural network to detecting onlookers. Accordingly, it would have been obvious to combine Coulon and Balavalikar Krishnamurthy to obtain claim 9.
Claim 19 recites a method with steps corresponding to the elements of the system recited in Claim 9. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Coulon and Balavalikar Krishnamurthy references, presented in rejection of Claim 9, apply to this claim.
Claims 10 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over David Coulon (US 2013/0021240 A1), in view of Balavalikar Krishnamurthy et al. (US 20240135748 A1), in further view of Maji et al. (US 20220277159 A1), and still in view of Harada et al. (US 20230023251 A1).
Regarding claim 10, the combination of Coulon and Balavalikar Krishnamurthy teaches, “information, (Coulon, Abstract discloses; “A method for controlling an electronic apparatus, includes steps of: acquiring an image of the environment of the apparatus, detecting the presence of human faces in the image acquired, estimating a respective position of each face detected in relation to the apparatus” The “presence of human faces” is interpreted to be face information.) confidence score information, (Balavalikar Krishnamurthy, Para. [0025] discloses; “Though not shown, it will be appreciated that the AI human head detector model 33 may also compute, for each head bounding box, a corresponding confidence measure or score which quantifies the model's confidence that a human head is detected.”) category information, (Coulon, Para. [0014] discloses; “According to one embodiment, the method comprises steps of determining whether or not each face detected in a field of view of the apparatus, is that of a user of the apparatus or of a person authorized by the user, or that of a potential intruder, based on the estimated distance between the apparatus and the detected face.” Intruder and user are category information.) angle information corresponding to the face information,” (Coulon, Para. [0011] discloses; “According to one embodiment, the estimation of the position of each face detected in an image comprises a step of estimating an angle formed between a camera axis direction of the image acquired and a direction in which the detected face is located.”) “corresponding one of the output feature tensors, to generate the information tensors, and each of the information tensors is configured to indicate … and key point information”, Examiner relies on the teachings of Maji in an analogous field of endeavor. Specifically, Maji teaches, “The onlooker detection system according to claim 9, wherein the neural network module comprises an output feature tensor generation module and a plurality of prediction modules, wherein the output feature tensor generation module is configured to generate a plurality of output feature tensors of different sizes based on the image, each of the prediction modules is configured to receive a corresponding one of the output feature tensors, to generate the information tensors” (Maji, Paras. [0014] - [0015] discloses; “The BFE module is configured to receive image data, or an image data set, that includes at least one object, and generate a number of feature tensors based on the image data. Each feature tensor has a different scale. In one embodiment, each feature tensor includes feature map data defined by a height, a width, and a number of channels, and the scale of the feature tensor is defined as height.Math.width. The FD module is configured to receive the feature tensors from the BFE module, perform a number of detection passes, and generate an intermediate tensor associated with each feature tensor. Each detection pass includes applying a random dropout mask to the feature tensors, and, for each feature tensor, determining scale-dependent bounding box spatial data that includes mean corner co-ordinates and covariance matrices for the corner co-ordinates for a number of scale-dependent bounding boxes for the object. Each intermediate tensor includes the scale-dependent bounding box spatial data for each detection pass for the associated feature tensor.).
Coulon, Balavalikar Krishnamurthy, and Maji are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Coulon and Balavalikar Krishnamurthy to incorporate the teachings of Maji in order to determine person information using a neural network. One would have been motivated to combine the previously described device of Coulon and Balavalikar Krishnamurthy with the teachings of Maji to apply a neural network to detecting onlookers. Accordingly, it would have been obvious to combine Coulon and Balavalikar Krishnamurthy to obtain the above specified limitations.
Further regarding claim 10, since the combination of Coulon, Balavalikar Krishnamurthy, and Maji does not explicitly teach, “and key point information”, Examiner relies on the teachings of Harada in an analogous field of endeavor. Specifically, Harada teaches, “and key point information” (Harada, Para. [0112] discloses; “Subsequently, the processor 122 estimates the presence or absence of the twist in the body of the person captured in the image I based on a ratio between the distance D1 and the distance D2. Specifically, when a difference between the ratio and 1 exceeds a threshold value, it is estimated that the body is twisted. When a person as the subject 30 faces the imaging device 11, it is highly probable that the left shoulder feature point LU1 and the right shoulder feature point RU1 are located symmetrically with respect to the face feature point F in the left-right direction (X direction).”)
Coulon, Balavalikar Krishnamurthy, Maji, and Harada are considered to be analogous to the claimed invention because they are in the same field of detecting people/faces in relation to a camera. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the combination of Coulon, Balavalikar Krishnamurthy, and Maji to incorporate the teachings of Harada in order to determine person information using a neural network. One would have been motivated to combine the previously described device of Coulon, Balavalikar Krishnamurthy, and Maji with the teachings of Harada to apply a neural network to detecting onlookers. Accordingly, it would have been obvious to combine Coulon and Balavalikar Krishnamurthy to obtain claim 10.
Claim 20 recites a method with steps corresponding to the elements of the system recited in Claim 10. Therefore, the recited steps of this claim are mapped to the proposed combination in the same manner as the corresponding elements in its corresponding system claim. Additionally, the rationale and motivation to combine the Coulon, Balavalikar Krishnamurthy, Maji, and Harada references, presented in rejection of Claim 10, apply to this claim.
Allowable Subject Matter
Claims 7 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable over prior art if: (a) rewritten in independent form including all of the limitations of the base claim and any intervening claims; and (b) the above-described rejection based on 35 U.S.C. 101, abstract idea is overcome.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUSTIN M. OAKES whose telephone number is (571)272-9379. The examiner can normally be reached 7:30am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at (571) 272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for
information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JUSTIN M OAKES/Examiner, Art Unit 2662
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights