DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is response to the preliminary amendment and application as filed on November 25, 2024. Claims 1-7 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on November 25, 2024 is in compliance with the provisions of 37 CFR § 1.97. Accordingly, the IDS has been considered by the examiner.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), it is being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. The acquisition means corresponds to a camera. Identification means, extraction means, storing means, and determination means correspond to a Central Processing Unit (CPU) implementing software.
If applicant does not intend to have this limitation interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation to avoid it being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation recite sufficient structure to perform the claimed function so as to avoid it being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. § 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-7 are rejected under 35 U.S.C. § 112(b), as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, regards as the invention.
With regard to claims 1-7, the claims are generally indefinite appearing to be a literal translation into English from a foreign document and are replete with grammatical and idiomatic errors. For example, instances of “gazed” should read --gaze-- and “visual field range” should read --field of view--. It is unclear what “gazed image” means. For example, is this an image of what the instructor or student is looking at, an indication in an image where the user focusing, fixated or otherwise concentrating on within the field of view or is this an image capturing the instructor’s or students gaze. For purposes of examination, this is interpreted as what the instructor or student is looking at. Sentence structure of the claims is also awkward and/or grammatically incorrect making reading and parsing of the claim difficult and time consuming. The claims should be reviewed and rewritten to remove any ambiguity.
In re claims 1 and 7, it is unclear what the coordinate information references, for example, is this the area within the video image that the user is looking at or is this information about the user’s eyes the user’s eyes for exam direction of the eyes and/or depth of focus or information about the area of focus within the video. For purposes of examination this is construed as coordinates with the video that is a focus of the user.
In re claims 2 and 3, the language “right/wrong” is indefinite. It is not clear what is meant by the “/” in this context. For example, it is unclear if the use of a slash is supposed to mean “or” or some other conjunction. For example, does the slash require a determination of whether the work is right or wrong or both. For purposes of examination this is construed as determining either right or wrong (but not both).
In re claims 2, 3, and 4 the claims recite the limitation “the gazed image.” This term lacks clear antecedent basis in these claims. There are a plurality of gazed images (e.g., instructor and worker) recited in the claim, and it is not clear which gazed image is “the gazed image” referred to here.
In re claim 2, the claim also recites the limitations “the work performed on a work target” at line 4. There is insufficient antecedent basis for this limitation in the claim. Claim 2 also recites “a database.” It is unclear where the database recited in claim 2 is the same as or different than the database recited in claim 1. Similarly, claim 2 recites “video information”; however, it is unclear from the claim whether this is video information of the gaze image of the worker or the instructor.
Claim 3 recites “a database.” It is unclear where the database recited in claim 3 is the same as or different than the databases recited in claims 1 and 2. In addition, claim 3 recites “the database.” This term lacks clear antecedent basis in the claim. There are a plurality of databases recited in the claims and it is not clear which database is “the database” referred to here. In addition, the recitation “the determination” at lines 6-7 also lacks clear antecedent basis as there are several determinations referred to in the claims.
In re claim 4, the claim recites “them” at line 5. It is unclear what “them” refers to. For example, does it refer to the response information or both the response information and gazed image together. In addition, it is unclear if the data set includes the response information, gazed image, and setting condition as the sentence structure and use of “including” is not clear.
In re claim 5, the claim recites “the video information” at lines 2, 5, and 10. This term lacks clear antecedent basis in the claim. The claim also recites “the display” at line 6. There is insufficient antecedent basis for this limitation in the claim.
Claim 6 recites “the database.” This term lacks clear antecedent basis in the claim. There are a plurality of databases recited in the claim and it is not clear which database is “the database” referred to here. The claim also recites the limitations “the attention data set” at lines 6, 10, and 17 and “the gaze target” at line 7. These terms lack clear antecedent basis in the claim. An attention data set and a gaze target are recited in claim 4; however, claim 6 depends from claim 2. The claim also recites the limitation “the selection” at lines 10. This term lacks clear antecedent basis in the claim. It is unclear whether the verb “select” only applies to “star of work” or also includes selecting of the attention display area and attention information. For purposes of examination this limitation is interpreted as selection of the start of work, the attention display area and the in the information.
In addition, the claim limitations directed to “extraction means,” “storing means,” “determining means,” and “display means” invoke 35 U.S.C. 112(f). The written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The means are interpreted as a processor implementing software to provide the claimed functions. However, the written description provides no corresponding software/algorithm for implementing the functions associated with these means in any detail other than the recitation of the function in the description. 1
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b).
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claims 2-6 depend from a rejected base claim and therefore are rejected for at least the reasons provided for the base claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. § 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4 and 7 are rejected under 35 U.S.C. § 102(a)(1) as being anticipated by U.S. Publication No. 2011/0111384 by Dietrich et al. (“Dietrich”).
In re claims 1 and 7, Dietrich discloses an attention extraction system and method for extracting attention information on work [Figs. 3-5, and Abstract, ¶¶46,51], comprising: acquisition means that acquires video information in a visual field range of an instructor who performs the work and coordinate information indicating a viewpoint that the instructor gazes at in the visual field range in time series order in association with the work [Figs. 1, 3, and 5, ¶¶46, 49, and 59, among others, describe a gaze tracker, including gaze tracking device 101 for tracking expert’s field of view performing a task including gaze position coordinates including video information]; identification means that identifies a gaze mode of the instructor as a wide visual field mode or a vigilant mode based on the video information and the coordinate information acquired by the acquisition means, the wide visual field mode being identified when the coordinate information in time series order indicating a viewpoint displacement of the instructor in the visual field range is aggregated in a center of the visual field range, the vigilant mode being identified when the coordinate information in time series order indicating the viewpoint displacement of the instructor in the visual field range is decentralized to an outside of the center of the visual field range [Fig. 3, ¶¶47-49, among others, describe the process examines the gaze coordinates for a focus pattern. These gaze coordinates indicate the position of the user's gaze over a given time (time series). A focus pattern occurs when the gaze coordinates are all within some boundary without gaze coordinates being statistically far from the others (determining a wide mode vs a vigilant)]; extraction means that sets a gazed region based on the gaze mode identified by the identification means, and extracts a gazed image of a work target that the instructor has gazed at in the gazed region [Figs. 3 and 5, ¶¶47-49, 54-59, among others describe determining a gaze region on an instructors focus while performing a task and storing the information for comparison to a novice]; and storing means that associates the gazed image extracted by the extraction means with the visual field range, the viewpoint displacement, and the gaze mode, and stores them in a database as attention information on the work [Fig. 4, 443, ¶¶24, 56, among others describe a database for storing instructor information for associated images or tasks].
In re claim 2, Dietrich discloses the acquisition means further includes: determination means that acquires a gazed image of a worker who performs work, and determines right/wrong of the work performed on a work target by the worker based on video information included in the gazed image and target information on work to be dealt with by the worker, the target information being stored in a database in advance; and display means that displays a determination result of the determination means [Figs. 1, 4-5, ¶¶54-59, among others, describe the gaze patterns of a novice performing the identical task via the GUI, are compared to the prior captured and stored gaze patterns of the expert via the GUI when viewing the same image. The differences between both gaze patterns are determined by well-known machine learning algorithms which differences are associated with significant outcomes (positive or negative outcomes) in real time].
In re claim 3, Dietrich discloses a database that stores an association between past gazed image information acquired in advance and reference information indicating the right/wrong of the work target associated with the gazed image, wherein the determination means refers to the database to determine the right/wrong of the work target, and acquires response information corresponding to a result of the determination from the database, and the display means further outputs the response information acquired by the determination means [Figs. 4-5, ¶¶24, 54-59, among others, describe The database, which is maintained for associated images or tasks, is a catalog of all the differences in behavior between the expert and the novice. The two primary aspects that would be watched for comparative purposes, in one example embodiment, are the differences in location (where the gaze is fixated) and duration (how long a given area gets attention), although the tracking of the gaze path may also be performed. differences between the two sets are determined and a database of these differences may then be created. The system thus tracks a novice's gaze pattern of that particular image while working and compares the novice's tracked gaze pattern to the database of captured and stored expert gaze patterns. When a deviation occurs, i.e., a gaze pattern is not fixated at an area that needs attention, or the gaze pattern indicates viewing along an incorrect path, e.g., a "negative" outcome (i.e., wrong), it is corrected through the user interface (the response)].
In re claim 4, Dietrich further comprising input means that inputs response information output by the determination means, wherein the storing means associates the response information input by the input means with the gazed image, and stores them as an attention data set including a setting condition for recognizing a gaze target [Figs. 4-5, ¶¶24, 54-59, among others describe The database, which is maintained for associated images or tasks, is a catalog of all the differences in behavior between the expert and the novice. The two primary aspects that would be watched for comparative purposes, in one example embodiment, are the differences in location (where the gaze is fixated) and duration (how long a given area gets attention), although the tracking of the gaze path may also be performed. Differences between the two sets are determined and a database of these differences may then be created. The system thus tracks a novice's gaze pattern of that particular image while working and compares the novice's tracked gaze pattern to the database of captured and stored expert gaze patterns. When a deviation occurs, i.e., a gaze pattern is not fixated at an area that needs attention, or the gaze pattern indicates viewing along an incorrect path, e.g., a "negative" outcome (i.e., wrong), it is corrected through the user interface (the response). Finally, the display is modified, in real-time, accordingly in order to correct, if necessary, the novice's behavior (e.g., if behavior is detected as tied to negative outcomes).].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. § 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 5 is rejected under 35 U.S.C. § 103 as being unpatentable over Dietrich in view of US Publication No. 2019/0332931 by Montantes (“Montantes”) and further in view of US Publication No. 2022/0019319 by Yamamoto et al. (“Yamamoto”).
In re claim 5, Dietrich discloses storing video information associated with the expert and novice in a database. Dietrick lacks, but Montantes teaches an attention extraction system, wherein the video information acquired by the acquisition means includes recording date and time information and recording location information of the work performed by the instructor, and recording control information regarding an acquisition operation of the video information [¶¶44-52, among others, describe creation of attention maps for experts/teachers performing a task including gaze information obtained through sensors for comparison with trainees/students. In particular, a particular set of sensor input data provided by a particular expert user that is provided during a particular input session. The set of sensor input data may be uniquely tagged (e.g., with the expert user login, time, date, etc.) such that the particular set of sensor input data is explicitly related to the particular expert user and the particular input session].
Dietrich and Montantes are both considered to be analogous to the claimed invention because they are in the same field of using expert gaze data to train users to perform a targeted task. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system/method of Dietrich to including storing the recording information acquired during the expert performing the task, as taught by Montantes, in order to provide improve collection and identification of expert responses, for example, by refining data to provide a deeper refinement of anomalous activity in an actual environment and remedies to the anomalous activity, see, e.g., ¶¶ 18, and 19.
Dietrich in view of Montantes teaches obtaining the recording date and time information, the recording location information, and the recording control information. However, Dietrich in view of Montantes does not explicitly teach displaying the information in a center of the visual field range before the acquisition of the video information by the acquisition means and switching the display to indicate only the recording control information at a corner of the visual field range during the acquisition of the video information.
However, Yamamoto teaches for a head-mounted display [¶22] and switching the display of information in a display screen from the center to a corner during operation of the display [¶32 describes the switchover manipulation guiding image G1 may be arranged at a position, on the outer circumference of the sub-screen image S, on a side close to an outer circumference of the main screen image M. In the example of FIG. 4, the sub-screen image S is arranged in the vicinity of the bottom right corner of the main screen image M, and further the switchover manipulation guiding image G1 is arranged at a bottom right corner of the sub-screen image S. In contrast, in a case where the sub-screen image S is arranged in the vicinity of a top left corner of the main screen image M, the switchover manipulation guiding image G1 may also be arranged at a top left corner of the sub-screen image S. If this is the case, the switchover manipulation guiding image G1 is arranged at a position away from the center of the main screen image M, so that the display of the main screen image M is less liable to be disturbed].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system/method of Dietrich in view of Montantes to including switching the display of information associated with a video (date, time, location etc.) from a central location to a corner while obtaining the expert or novice data, as taught by Yamamoto, in order to provide improved user experience, for example, so the display of the information is less like to disturb the user during recording, see, e.g., ¶32.
Examiner’s Note
In re claim 6, the prior art of record, does not anticipate of obviate the following recitations “the display means further includes an attention information display area that displays an acquisition display area and an attention display area to the worker who performs the work while the acquisition display area and the attention display area are mutually switched, the acquisition display area indicating a type of the acquisition means that acquires the video information, a type of the attention data set stored in the database and including a setting condition for recognizing the gaze target, and an instruction for causing each worker to select a start of the work, the attention display area indicating response information corresponding to the gazed image of the worker and attention information based on the attention data set after the selection, the attention information indicated in the attention display area includes nudge information indicated at a timing of at least any of a work start, a middle of the work, or a work end corresponding to work progress of the worker, and information including at least any of the response information, the attention information, or the nudge information indicated in the attention information display area is distributed to be displayed depending on a gaze state of the worker who performs the work based on the attention data set and the determination result of the determination means,” in combination with the rest of the elements in claims 6, 2, and 1.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and is listed on the attached Notice of References Cited.
US 2016/0027336 describes using an eye tracker device, monitoring a pilot's gaze during operation of the aircraft; detecting a gaze location where the pilot's gaze intersects an instrument panel or a flight deck window of the aircraft during operation of the aircraft; and storing gaze location data in a data storage device onboard the aircraft during operation of the aircraft. The data storage device may be a flight data recorder. The system also describes retrieving stored gaze location data from the data storage device; performing a comparative analysis comparing the retrieved gaze location data with baseline standard data; and assessing pilot performance based on results of the comparative analysis comparing the retrieved gaze location data with expert baseline standard data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Andrew Bodendorf whose telephone number is (571) 272-6152. The examiner can normally be reached M-F 9AM-5PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xuan Thai can be reached on (571) 272-7147. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW BODENDORF/Examiner, Art Unit 3715
/XUAN M THAI/Supervisory Patent Examiner, Art Unit 3715
1 See, e.g., MPEP 2181 II B & C wherein a rejection under 35 U.S.C. 112(b) is appropriate if the specification discloses no corresponding algorithm associated with a computer or microprocessor. Aristocrat, 521 F.3d at 1337-38, 86 USPQ2d at 1242. See also, Blackboard, Inc. v. Desire2Learn, Inc., 574 F.3d 1371, 1382-83, 91 USPQ2d 1481, 1490-91 (Fed. Cir. 2009) (concluding that the description of a server computer’s "access control manager" software feature was insufficient disclosure of corresponding structure to support the computer-implemented "means for assigning" limitation because "what the patent calls the ‘access control manager’ is simply an abstraction that describes the function of controlling access to course materials … [b]ut how it does so is left undisclosed."); Aristocrat, 521 F.3d at 1334-35, 86 USPQ2d at 1240 (explaining that "the [patent’s] description of the embodiments is simply a description of the outcome of the claimed functions, not a description of the structure, i.e., the computer programmed to execute a particular algorithm").
Prosecution Insights