DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
i. Claims 1 – 3, 5 – 24 are rejected under 35 U.S.C. 103 as being unpatentable over Naidoo et al. (2021/0103262; hereinafter Maria)1 in view of Le (2015/0205398)2.
Regarding claim 1, Maria discloses one or more tangible, non-transitory computer readable media storing machine-readable instructions that, when executed by one or more processors [0162] of user device [0074], cause the user device to:
receive one or more gestures, the one or more gestures being provided by a user ([0078]: User interface comprising components for gesture input);
provide the one or more gestures as one or more inputs (User interactions defining logic [0099] of controller configuration with template blocks [0098]) to identify, a process entity, the process entity being identified from among a plurality of process entities having respective template objects associated therewith (Selected one among template objects representing piece of equipment [0111] of a process plant [0048]); and
based upon the identification of the process entity, display, in a canvas region of a graphical user interface (GUI) executing via the user device, of an instance of a template object corresponding to the identified process entity (Graphics of corresponding equipment dragged by user into canvas area of GUI [0112] that is displayed [0094]), the template object being configured to
provide a visual representation of operation of the identified process entity in a process plant to thereby form at least a portion of a process display (GUI representing area of a plant comprising equipment [0088] whose parameters may be monitored and changed [0071]).
Maria does not explicitly disclose the media configured to receive one or more gestures forming at least one line or shape in a gesture input region; provide the one or more gestures as one or more inputs to a gesture recognition algorithm to identify, based at least upon the received one or more gestures, a process entity represented by the received one or more gestures.
In the same field of endeavor, Le discloses graphical object management [0005] configured to receive one or more gestures [0006] forming at least one line or shape [0021] in a gesture input region, the one or more gestures being provided by a user [0033]; provide the one or more gestures [0006] as one or more inputs to a gesture recognition [0089] algorithm [0046] to identify, based at least upon the received one or more gestures, a process entity represented by the received one or more gestures ([0021]: Process control shapes). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified as configured to receive one or more gestures forming at least one line or shape in a gesture input region; provide the one or more gestures as one or more inputs to a gesture recognition algorithm to identify, based at least upon the received one or more gestures, a process entity represented by the received one or more gestures, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 2, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein receiving the one or more gestures comprises detecting the one or more gestures via one or more cameras associated with the user device [0078].
Regarding claim 3, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the gesture input region is the canvas region of the GUI [0098], and wherein the one or more gestures are received via one or more touch interactions corresponding to the canvas region (Canvas area implemented in electronic device [0088] whose input components comprise touch sensors [0071]).
Regarding claim 5, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1.
Maria does not explicitly disclose the media wherein the instructions, when executed by the one or more processors, further cause the user device to display the at least one line or shape in the canvas region of the GUI upon receiving the one or more gestures.
In the same field of endeavor, Le discloses graphical object management [0005] wherein the instructions, when executed by the one or more processors, further cause the user device to display the at least one line or shape in the canvas region of the GUI upon receiving the one or more gestures ([0025]: Output of appropriate graphical object in response to sensed motion/gesture). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the instructions, when executed by the one or more processors, further cause the user device to display the at least one line or shape in the canvas region of the GUI upon receiving the one or more gestures, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 6, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 5.
Maria does not explicitly disclose the media wherein the instructions, when executed by the one or more processors, further cause the user device to display an icon selectable to cause the one or more gestures to be provided to the gesture recognition algorithm, and wherein the providing of the one or more gestures to the gesture recognition algorithm is performed responsive to receiving a selection of the icon via the GUI.
In the same field of endeavor, Le discloses graphical object management [0005] wherein the instructions, when executed by the one or more processors, further cause the user device to display an icon selectable to cause the one or more gestures to be provided to the gesture recognition algorithm, and wherein the providing of the one or more gestures to the gesture recognition algorithm is performed responsive to receiving a selection of the icon via the GUI ([0113]: Prompt comprising shape options, in response to user’s drawn gesture). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the instructions, when executed by the one or more processors, further cause the user device to display an icon selectable to cause the one or more gestures to be provided to the gesture recognition algorithm, and wherein the providing of the one or more gestures to the gesture recognition algorithm is performed responsive to receiving a selection of the icon via the GUI, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 7, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the instructions to provide the one or more gestures to the gesture recognition algorithm include instructions to: receive, from the server over the network, an indication of the identified process entity (Templates identifying respective ones among equipment objects stored by database [0088] implemented by server [0183]).
Maria does not explicitly disclose the media wherein the instructions to provide the one or more gestures to the gesture recognition algorithm include instructions to: transmit an indication of the one or more gestures over a network to one or more servers, the one or more servers being configured to implement the gesture recognition algorithm.
In the same field of endeavor, Le discloses graphical object management [0005] wherein the instructions to provide the one or more gestures to the gesture recognition algorithm include instructions to: transmit an indication of the one or more gestures over a network to one or more servers, the one or more servers being configured to implement the gesture recognition algorithm (Object prediction provided by manager [0029] transmitted by a server [0040]); and receive, from the server over the network, an indication of the identified process entity (Object prediction capable of being implemented in context [0112] of process control shapes [0021]). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the instructions to provide the one or more gestures to the gesture recognition algorithm include instructions to: transmit an indication of the one or more gestures over a network to one or more servers, the one or more servers being configured to implement the gesture recognition algorithm, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 8, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the instructions, when executed by the one or more processors, further cause the user device to display, via the GUI, a visual element indicating the process identity being identified, and wherein the displaying of the instance of the template object in the canvas region is performed responsive to receiving a selection of the visual element by the user via the GUI ([0088]: Graphic representing type of equipment, dragged and dropped onto canvas).
Maria does not explicitly disclose the media wherein the process identity being identified via the gesture recognition algorithm.
In the same field of endeavor, Le discloses graphical object management [0005] wherein the process identity being identified via the gesture recognition algorithm (Object prediction capable of being implemented in context [0112] of process control shapes [0021]). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the process identity being identified via the gesture recognition algorithm, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 9, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1.
Maria does not explicitly disclose the media wherein identifying the process entity includes: obtaining, via the gesture recognition algorithm, an indication of two or more candidate process entities; displaying, via the GUI, a menu indicating the two or more candidate process entities; receiving, from the user via the menu, a selection of a particular process entity from among the two or more candidate process entities; and identifying the selected process entity as the process entity represented by the one or more gestures.
In the same field of endeavor, Le discloses graphical object management [0005] wherein identifying the process entity includes: obtaining, via the gesture recognition algorithm, an indication of two or more candidate process entities ([0092]: Gesture coordinates are basis for object prediction); displaying, via the GUI, a menu indicating the two or more candidate process entities (User prompted with box containing predicted objects attempted [0113], [0114] in context of process control shapes [0021]); receiving, from the user via the menu, a selection of a particular process entity from among the two or more candidate process entities (From among objects included in prompt [0113], [0114] user selects one [0102]); and identifying the selected process entity as the process entity represented by the one or more gestures ([0102]: Upon user selection, formalized object is display). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein identifying the process entity includes: obtaining, via the gesture recognition algorithm, an indication of two or more candidate process entities; displaying, via the GUI, a menu indicating the two or more candidate process entities; receiving, from the user via the menu, a selection of a particular process entity from among the two or more candidate process entities; and identifying the selected process entity as the process entity represented by the one or more gestures, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 10, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 9.
Maria does not explicitly disclose the media wherein the instructions, when executed by the one or more processors, further cause the user device to cause the gesture recognition algorithm to be adapted based upon the selection of the particular process entity from among the two or more candidate process entities.
In the same field of endeavor, Le discloses graphical object management [0005] wherein the instructions, when executed by the one or more processors, further cause the user device to cause the gesture recognition algorithm to be adapted based upon the selection of the particular process entity from among the two or more candidate process entities (In context of process control shapes [0021] user selection among prompted [0102] object options [0113] may form basis for user-specific adaptation [0042], [0091]). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the instructions, when executed by the one or more processors, further cause the user device to cause the gesture recognition algorithm to be adapted based upon the selection of the particular process entity from among the two or more candidate process entities, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 11, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1.
Maria does not explicitly disclose the media wherein the identification of the process entity by the gesture recognition algorithm is further based upon a location in the gesture input region at which the one or more gestures were received, with respect to locations of one or more other template object instances in the canvas region.
In the same field of endeavor, Le discloses graphical object management [0005] wherein the identification of the process entity by the gesture recognition algorithm is further based upon a location in the gesture input region at which the one or more gestures were received, with respect to locations of one or more other template object instances in the canvas region (In the context of process control shapes [0021] thresholds partition linked gestures into respective graphical objects, without a user lifting their finger [0023]). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the identification of the process entity by the gesture recognition algorithm is further based upon a location in the gesture input region at which the one or more gestures were received, with respect to locations of one or more other template object instances in the canvas region, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 12, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein displaying the instance of the template object in the canvas region comprises automatically adjusting a size or location of the instance of the template object in the canvas region ([0129]: User dragging template to canvas automatically places input blocks in input area).
Regarding claim 13, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 12.
Maria does not explicitly disclose the media wherein automatically adjusting the size or location of the instance of the template object comprises snapping an edge of the instance of the template object to at least one other template object instance in the canvas region.
In the same field of endeavor, Le discloses graphical object management [0005] wherein automatically adjusting the size or location of the instance of the template object comprises snapping an edge of the instance of the template object to at least one other template object instance in the canvas region (Detection of gesture coordinates within threshold distance of one of existing graphical objects [0087] aligns connector endpoints with edges of the graphical objects [0100]). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein automatically adjusting the size or location of the instance of the template object comprises snapping an edge of the instance of the template object to at least one other template object instance in the canvas region, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 14, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the instructions, when executed via the one or more processors, further cause the user device to: display a configuration region of the GUI [0149]; and receive, via the configuration region of the GUI, a plurality of user interactions to associate the process display with process entities included at least a portion of the process plant [0071], to facilitate operation of the process display using on-line process data from the at least the portion of the process plant [0088].
Regarding claim 15, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the instructions, when executed via the one or more processors, further cause the user device to upload the process display to one or more servers (User uploads configuration to database [0094] implemented by server [0199]).
Regarding claim 16, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the instructions, when executed via the one or more processors, further cause the user device to download the process display to one or more other devices for viewing of the process display at the one or more other devices during on-line operation of the process plant [0076].
Regarding claim 17, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the instructions, when executed via the one or more processors, further cause the user device to define, based upon one or more user interactions received via the GUI, a new association between one or more user-defined lines or shapes and a particular process entity having a corresponding template (Modifying configuration [0094] including control routine parameters of [0127] equipment [0106]).
Regarding claim 18, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 17.
Maria does not explicitly disclose the media wherein the instructions, when executed via the one or more processors, further cause the user device to cause the gesture recognition algorithm to be adapted to recognize the one or more user-defined lines or shapes as corresponding to the particular process entity.
In the same field of endeavor, Le discloses graphical object management [0005] wherein the instructions, when executed via the one or more processors, further cause the user device to cause the gesture recognition algorithm to be adapted to recognize the one or more user-defined lines or shapes as corresponding to the particular process entity (Users’ gestures and feedback [0043] inform the object predictor [0091] implemented for process control shapes [0021]). This is among measures implemented to overcome the difficulty of drawing shapes on a low-friction device surface [0004].
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the instructions, when executed via the one or more processors, further cause the user device to cause the gesture recognition algorithm to be adapted to recognize the one or more user-defined lines or shapes as corresponding to the particular process entity, in view of the teaching of Le, to overcome the difficulty of drawing shapes on a low-friction device surface.
Regarding claim 19, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 1. Maria discloses the media wherein the instructions, when executed via the one or more processors, further cause the user device to modify one or more properties of the displayed instance of the template object in response to one or more further user interactions received via the GUI (Modifying configuration [0094] including control routine parameters of [0127] equipment [0106]).
Regarding claim 20, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 19. Maria discloses the media wherein the one or more properties include an animation ([0152]: User configuring equipment’s respective graphics to display with color that is context/parameter specific).
Regarding claim 21, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 19. Maria discloses the media wherein the one or more properties include an event handler behavior ([0044]: Previewing control logic for controlling a process).
Regarding claim 22, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 19. Maria discloses the media wherein the one or more properties include a physical dimension [0155].
Method claim 23 and device claim 24 are rejected as reciting limitations similar to those recited in media claim 1.
ii. Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Maria in view of Le, as applied to claim 3 above, and further in view of Hatano et al. (2019/0332242; hereinafter Hatano).
Regarding claim 4, Maria in view of Le discloses the one or more tangible, non-transitory computer readable media of claim 3. Maria discloses the media wherein the canvas region is displayed via a visual display of the user device ([0130], [0132]), and wherein the one or more gestures are received via the one or more touch interactions at a touch-aware surface of the user device ([0089]: e.g. touchscreen).
Maria in view of Le does not explicitly disclose the media wherein the touch-aware surface is separate from the visual display of the user device, and wherein receiving the one or more gestures comprises mapping respective locations of the one or more touch interactions at the touch-aware surface to a corresponding respective locations of the visual display.
In the same field of endeavor, Hatano discloses a coordinate input device [0002] wherein the touch-aware surface (Figure 1: Comprising 2) is separate from the visual display (Comprising 4) of the user device (Comprising 1), and wherein receiving the one or more gestures comprises mapping respective locations of the one or more touch interactions at the touch-aware surface to a corresponding respective locations of the visual display [0043]. This implementation avoids the strangeness of a smaller touch-pad interface ([0014], [0016]).
It would be obvious to one having ordinary skill in the art before the filing date of the claimed invention for the media of Maria to be modified wherein the touch-aware surface is separate from the visual display of the user device, and wherein receiving the one or more gestures comprises mapping respective locations of the one or more touch interactions at the touch-aware surface to a corresponding respective locations of the visual display, in view of the teaching of Hatano, to furnish natural interface navigation.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Aaron Midkiff whose telephone number is (571)270-5875. The examiner can normally be reached Monday - Friday, 8:00am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amr Awad can be reached at (571)272-7764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON MIDKIFF/
Examiner, Art Unit 2621
/AMR A AWAD/Supervisory Patent Examiner, Art Unit 2621
1 Cited in Applicant’s 15 October 2025 IDS.
2 Cited in Applicant’s 15 October 2025 IDS.